Load Wizard

The Custodian load wizard is available from the feeds menu item, the wizard gives the ability to load files of different formats from different locations.

The user can select the location of the file, this can be locally uploadable from the users machine, on the local network, or on a web server or ftp server

Http files can optionally specify a username and password to authenticate with simple authentication, plus you can also use HTTP headers if a specific security token needs to be passed to the server.

FTP downloads can optionally include a username and password for the FTP server, plus you may need to specify the FTP mode as Binary or ASCII depending on the line endings of the file.

All file types will be loaded into a new table unless the user specifies an existing table name to load the data into.

XML files require the XML Root Node to be specified, an XML root node is the XML tag that specifies the repeating record in the file that represents the data you want to load.

Excel spreadsheets can be loaded, but you need to specify the sheet number (rather than sheet name) for the loader if there are multiple sheets in the file.

CSV files or other delimited files can be loaded, you may specify the delimiter and quote characters for the file.

For all file type you may also skip a number of records or limit the load to a first section of the file. Please be aware that any installation in a trial mode will be limited to 50,000 records per load, unless the license for the server is extended to a higher amount or the server is registered as an Enterprise license.

The file will be inspected and the list of columns found will be displayed, you must specify the field that represents the primary key of the data source, if you do not specify this one will be created for you, however in the event that you load the same file twice, you may find the data is not updated as there is no key to identify whether the record exists or not. Later on in in the wizard you may specify how to create the ID for records without on.
One or more label fields must be selected to identify how to display the record in search results.
If the table already exists then these selections will be made for you.

Map your data columns in the file to the fuzzy concepts directly, again if this has been done before the screen will be populated for you. You may need to map the physical fields to the concept fields for multiple concepts.

You can save the load process if you specify a name for the Feed Template, it will then be available on the feed mange, Run and Schedule screens.

Specify a file name pattern if you wish to automatically pick up a varying file name from a location (ie HTTP or FTP server), you can use date and time patterns as per the following specification.

A pattern is defined starting with a $ and ending with a ; ie:

Example Pattern
$YYYYMMdd; translates to20010704
$yyyyy.MMMMM.dd GGG hh:mm aaa; 02001.July.04 AD 12:08 PM
EEE, d MMM yyyy HH:mm:ss Z Wed, 4 Jul 2001 12:08:56 -0700
yyMMddHHmmssZ 010704120856-0700

Each pattern is defined using:

LetterDate or Time ComponentPresentationExamples
GEra designatorTextAD
YYearYear1996; 96
MMonth in yearMonthJuly; Jul; VII; 07; 7
wWeek in yearNumber27
WWeek in monthNumber2
DDay in yearNumber189
dDay in monthNumber10
FDay of week in monthNumber2
EDay in weekTextTuesday; Tue
aAm/pm markerTextPM
HHour in day (0-23)Number0
kHour in day (1-24)Number24
KHour in am/pm (0-11)Number0
hHour in am/pm (1-12)Number12
mMinute in hourNumber30
sSecond in minuteNumber55
SMillisecondNumber970
zTime zoneGeneral time zonePacific Standard Time; PST; GMT-08:00
ZTime zoneRFC 822 time zone-0800
Escape for text/idDelimiter(none)
Single quoteLiteral

If the file does not have a primary key, one can be generated for you:

Use a sequence will generate a numeric sequence for each record, this is not recommended as it would cause duplicates if the file was loaded more than once.

Generate one from the label – will use the label data to generate a key for the record, this would mean that the same record loaded twice with the same label fields would cause an update the second time.

Generate one from the whole record will use a stamp of the whole record to generate an id, this is more expensive, and if one value changes the id generated would be different.

If you don’t wish to load the file now, slide the final button to say “No” – or leave it as Yes to start the load after the Next button is pressed.