Table of Contents
In many cases it is necessary to load a file from the server rather that using the user interface to define a feed and to schedule it. monsterDB provides 2 integration points for this:
- The Java API incorporating a bulk loader,
- Load a file directly from the command prompt.
This document discusses the monsterDB command prompt loader.
Loadfile
LoadFile is currently available via the Command Line (ie windows CMD or *nix bash) only:
java -jar monsterDB.jar -h host -p port -u username -pw password -d aDatabase -c aCollection -f file -r recordType -o options
Where aCollection is a collection (domain context) that can have existing documents within it.
RecordType is the custodian tablename, if you omit this, the record will not be matchable.
Filename/url a local (to the server you are on!) file that can be read by the current user, or a URL such as http://… https://… or ftp:.//… pointing at a file or zip file on the local network or accessible on the internet.
options are a quoted text string containing a json object, property values for this are:
- encoding – file character encoding, defaults to UTF8
- rootNode – in XML this will be used to extract the documents, every XML element that is named with this will be converted to a document.
- extractPattern – when a zip file is found by the URL then this pattern will . be used to extract the files needed from within it.
- quote – The character used to quote fields of data that might contain the delimiter.
- delimiter -delimiter character(s) between fields in the rows of data in a delimited file – ie ,
- fileType – ie CSV, XML, XLS which indicates the type of the file that will eventually be read.
- sheetno – with an XLS sheet this is the 0 based sheet number to read.
Plus if the file is a URL starting http, https or ftp then the following options can be used
- httpusername – username used to login to the remote http server
- httppassword – associated http password
- httpheaders – any headers required to create a http connection
- ftpusername – ftp username to login to the remote server
- ftppassword – ftp password
The following is an example to load a local XML file:
java -jar monsterDB.jar -h host -p port -u username -pw password -d aDatabase -c fuzzyCollection -f leismall.xml -r GLEIF -o "{fileType: "XML", rootNode: "LEIRecord"}"
And to load a XML file that has been ZIPPED and placed on a website:
java -jar monsterDB.jar -d newdb -c fuzzy2 -t GLEIF -f https://leidata.gleif.org/api/v1/concatenated-files/lei2/20190805/zip -o "{"fileType": "zip", "rootNode": "LEIRecord", "extractPattern": ".*.xml"}"