The Custodian load wizard is available from the feeds menu item, the wizard gives the ability to load files of different formats from different locations.
The user can select the location of the file, this can be locally uploadable from the users machine, on the local network, or on a web server or ftp server
Http files can optionally specify a username and password to authenticate with simple authentication, plus you can also use HTTP headers if a specific security token needs to be passed to the server.
FTP downloads can optionally include a username and password for the FTP server, plus you may need to specify the FTP mode as Binary or ASCII depending on the line endings of the file.
All file types will be loaded into a new table unless the user specifies an existing table name to load the data into.
XML files require the XML Root Node to be specified, an XML root node is the XML tag that specifies the repeating record in the file that represents the data you want to load.
Excel spreadsheets can be loaded, but you need to specify the sheet number (rather than sheet name) for the loader if there are multiple sheets in the file.
CSV files or other delimited files can be loaded, you may specify the delimiter and quote characters for the file.
For all file type you may also skip a number of records or limit the load to a first section of the file. Please be aware that any installation in a trial mode will be limited to 50,000 records per load, unless the license for the server is extended to a higher amount or the server is registered as an Enterprise license.
The file will be inspected and the list of columns found will be displayed, you must specify the field that represents the primary key of the data source, if you do not specify this one will be created for you, however in the event that you load the same file twice, you may find the data is not updated as there is no key to identify whether the record exists or not. Later on in in the wizard you may specify how to create the ID for records without on.
One or more label fields must be selected to identify how to display the record in search results.
If the table already exists then these selections will be made for you.
Map your data columns in the file to the fuzzy concepts directly, again if this has been done before the screen will be populated for you. You may need to map the physical fields to the concept fields for multiple concepts.
You can save the load process if you specify a name for the Feed Template, it will then be available on the feed mange, Run and Schedule screens.
Specify a file name pattern if you wish to automatically pick up a varying file name from a location (ie HTTP or FTP server), you can use date and time patterns as per the following specification.
A pattern is defined starting with a $ and ending with a ; ie:
| Example Pattern | |
|---|---|
| $YYYYMMdd; translates to | 20010704 |
| $yyyyy.MMMMM.dd GGG hh:mm aaa; | 02001.July.04 AD 12:08 PM |
| EEE, d MMM yyyy HH:mm:ss Z | Wed, 4 Jul 2001 12:08:56 -0700 |
| yyMMddHHmmssZ | 010704120856-0700 |
Each pattern is defined using:
| Letter | Date or Time Component | Presentation | Examples |
|---|---|---|---|
| G | Era designator | Text | AD |
| Y | Year | Year | 1996; 96 |
| M | Month in year | Month | July; Jul; VII; 07; 7 |
| w | Week in year | Number | 27 |
| W | Week in month | Number | 2 |
| D | Day in year | Number | 189 |
| d | Day in month | Number | 10 |
| F | Day of week in month | Number | 2 |
| E | Day in week | Text | Tuesday; Tue |
| a | Am/pm marker | Text | PM |
| H | Hour in day (0-23) | Number | 0 |
| k | Hour in day (1-24) | Number | 24 |
| K | Hour in am/pm (0-11) | Number | 0 |
| h | Hour in am/pm (1-12) | Number | 12 |
| m | Minute in hour | Number | 30 |
| s | Second in minute | Number | 55 |
| S | Millisecond | Number | 970 |
| z | Time zone | General time zone | Pacific Standard Time; PST; GMT-08:00 |
| Z | Time zone | RFC 822 time zone | -0800 |
| ‘ | Escape for text/id | Delimiter | (none) |
| ” | Single quote | Literal | ‘ |
If the file does not have a primary key, one can be generated for you:
Use a sequence will generate a numeric sequence for each record, this is not recommended as it would cause duplicates if the file was loaded more than once.
Generate one from the label – will use the label data to generate a key for the record, this would mean that the same record loaded twice with the same label fields would cause an update the second time.
Generate one from the whole record will use a stamp of the whole record to generate an id, this is more expensive, and if one value changes the id generated would be different.
If you don’t wish to load the file now, slide the final button to say “No” – or leave it as Yes to start the load after the Next button is pressed.