Importing CSV Files via File Masks

When you import CSV files from Dropbox, FTP, and SFTP, you can specify a file mask using a date/time template instead of selecting a file. When package runs, it will search the file to import by substituting the current date to the mask.

With this functionality you can, for example, configure a regular upload of files to Dropbox and schedule Import package for automatic execution so that it imports a new CSV file every time, and you don't need to overwrite old files. All the files must have the same set of columns.

ExpandedToggleIcon        File Mask Format

The file mask uses the following format: a name fragment enclosed in braces "{}" is considered a date/time format string. When package runs, the date/time of the run is converted to the specified timezone, the specified date/time format string is applied to it, and the result string is substituted for the name fragment enclosed in braces.

note Note

The date/time format string is specified in the same format and applied in the same way as .NET Framework custom date and time format strings.

For example, if the following mask "export_{MM_dd_yyyy}.csv" is used, when a package is executed on August 18, 2018, it will try to load a file export_08_18_2018.csv. If no such file is found, the package run will fail.

It's not recommended to use seconds or fractions of seconds, and even minutes in the mask, since it's not guaranteed that package will run in exactly the specified time. When Skyvia is under peak load, package run may be delayed for a few seconds or even minutes. And even if the difference of the package run time and the date/time in the file name is minor, the package run will fail.

 

ExpandedToggleIcon        How to Use File Mask

To use the file mask, on the Source Definition page of the Import Wizard, select the Use File Mask check box, select the folder to load CSV files from, specify the File Mask and timezone to convert the package run date/time to before substituting to the mask.

import-file-mask

note Note

To expand a folder in order to select its subfolder, click the icon of this folder in the list.

After you have specified the folder, file mask, and timezone, click Apply.

Please note that Skyvia requires a CSV file with the name corresponding to the file mask with the current date/time (when you click Apply) to be present in the specified folder. This file is used to determine the columns, which will be used in mapping. All the files that will be loaded by the import task must have the same set of columns as this file, used on task creation. Without clicking Apply and loading the file to determine the columns, you cannot proceed to further import task editing.

If you need to change the set of columns for an existing import task, you can edit the task and click Apply again. The list of source columns will be reloaded from the file with the name corresponding to the current date/time.

If you want import a CSV file together with a ZIP archive with binary files, you need to specify the file mask, folder, and timezone for ZIP files separately. Click Add zip with binary data and then select the folder to load ZIP files from, specify the File Mask and timezone to convert the package run date/time to before substituting to the mask. Then click Apply.

csv-zip-mask

After you have configured your file mask settings, proceed to setting up your target and mapping settings and save the task.