Table of Contents
Metadata is a guidance rather than a rule.
There are multiple ways to build a table, each result in the same thing, metadata. Although the match rules and the user input screens in Custodian require metadata in the form of a table and columns, the matching system treats it more as a guidance rather than an essential. As such it is actually possible to store more information in a record (document) representing a master data entity such as product, customer etc than is mentioned in the metadata. This somewhat future proofs the situation enabling you to be able to load data that is essentially invisible to the end user, but can be carried forward to the systems downstream of EntityStream Custodian.
Considerations in your Model
Additionally as metadata in can be based upon the source data or based upon a golden record type model, and depending on your approach to Master Data and how you will deploy the system then this will direct you to one approach or the other.
Generally speaking if you require to author (as in originate data) data from Custodian then you will want to choose a golden record type model, if not and you are looking to link together records from many systems into a global cross reference then you will want to define the models closer to that of the source system.
| Golden Record Model | Source Model |
|---|---|
| Single Model Representing all Systems | One Table per data source per system |
| Merging, Linking or EID possible | Linking or EID possible |
| One set of Concept Mappings | One Concept Mapping per Source |
| Suitable for Full Authoring of new Data | Authoring Suitable for Maintenance only |
| Harder to keep in line with all systems | Simpler to maintain, add new sources etc |
| Integration requirements are heavy (M:1) | Less Integration Needed (1:1) |
The approach you take is completely up to you, but often it is harder to change strategy later on, so a modicum of thought at the start is preferred.
Creating a Table and its Columns.
Use the Data Model tool to create a table definition, use the drop down button on the top right to do this, once you have created your table and at least 1 column then you can save the table definition using the blue button at the bottom right.
Add a table using the menu options, once you have given the table a name at the prompt, it will be checked to ensure that you have not created one of this name before and then allow you to create columns. Please note that it is possible to create two tables of the same name IF they are in different Domains (contexts), this is by design, but it can cause some confusion to the end user as the descriptive name for the table will be the same unless you change the table display name under the settings. The display name is highlighted for you.
If you expand the table properties using the down chevron, then the following options as displayed:
| Option | Meaning |
|---|---|
| Display Name | User friendly table name, you can use all characters |
| Icon URL | locally or publicly accessible image file (icon size 16×16) or image class names such as ‘fa fa-hand’ |
| Audited Table | Checkbox to store timestamped changes for each document in table. |
| Date and Time format of Source (ie uuuu-MM-dd’T’HH:mm:ss[.SSS]XXX) | If the table contains dates and you want to convert them to proper dates for storage, then this pattern needs to be set |
| Represents Internal Data | No longer used |
| Documented as Master Data Table | Does this table have significance, it causes the dashboard to show the statistics for this table. |
| Use for Relationship Payload ONLY | If this table represents a relationship table to store attributes for a hierarchy, then this option should be chosen, if it is chosen then the sub option will be available: “Restricted Relationships to one Table”. Restricted relationship table can only store relationships to-from the same table type ie Person-Person. Unrestricted allow any relationships between table types such as company-person. |
| Maximum number of records to compare # | When matching, this option be used to prevent over-matching within the table. 1000 records would be a potential 500,000 comparisons and this number prevents too many calculations for scoring when the data is very “matchy” |
| Maximum number of records to compare % | Again when matching, this %age will prevent over-matching on smaller data sets where the numeric value above would be less effective. |
| Published Table | Not currently used (reserved for future) |
Special Attention should be paid to the “Documented as a Master Table” and, “Use for Relationship Payload ONLY”.
Once the table has been created you must add at least 1 column, you can either choose to use the add column option which will simply create a new column, but you will need to over type the values for name and display name. A more efficient way to create columns is to use the paste text option:
You may now enter the column names into the window as a comma separated list, please be careful with spaces, spaces will be used too, so a good example is:
Brand ID,Brand Name,Brand Name (Short),Definition,Comment
You may also enter a json object:
{"Brand ID":"text", "Brand Name":"text", "Definition Date": "date"}
The column properties are changeable using the right hand side of the screen, their meaning is:
| Option | Meaning/Values |
|---|---|
| Column Name | The physical name of the column |
| Display Name | The user friendly name to use in the search, authoring, etc |
| Primary Key | Does this column represent a primary key |
| Position in Label | Each record needs a label field, this position of the column in the label enables you to define a concatenated label |
| Validation Pattern | REGEX (regular expression) to validate and reject/suspend the data |
| Display Type | Text List (array) Structure (ie nested item) Date Time Numeric Boolean Image Each other Table will also appear to enable you to pick a table from any domain as a foreign key reference. |
| Display Page | The page on the search and edit screen to place this field |
| Merge Action | The action to take when contructing a golden record incorporating this field. Use Latest Use Earliest Use Longest |
| Default Value | The default value to use when no value is supplied by the source/user |
| Auto Generated Sequence | If the field needs to be populated with a sequence then this option can be chosen. |
Once a column is created, you can also create a sub column provided the column was defined as a Structure. This will enable you to create nested recprd structures. Also a list of items can be created to represent an array of items.
You will need to ensure that one primary key field is chosen and at least one label field is defined before saving the table.