Defining Rules and Rule Mappings for a OneData Matching Project
Use the following procedure to create new rules and rule mappings for OneData Matching using the Manage DQ Rules option.
By default, every project in
OneData has a default
OneData Matcher project with a set of rules defined. You can use this default project for matching or add create a new project, add rules and configure it in accordance with your data quality model. For details on the default rules, see
The Default
OneData Matcher Rules and Attributes.
1. On the Menu toolbar, click Define > Data Quality > Consolidation > Project Mapping.
2. In Filter by Project, select the OneData matching project.
3. Click Manage DQ Rules.
If you had selected Inherit Rules from Default Project when creating the OneData project, the rules from the default OneData Matcher project are automatically associated with the project. The Rules Management screen displays rules (associated with the project) separated into BOTH and MASTER pattern grids. The associated rules are displayed first by order of priority, followed by the rules without any associations.
4. In the relevant pattern grid (BOTH or MASTER) to which you want to add the new rule, click Add Rule.
If the rules are added within the context of a project, OneData automatically associates the rules to the project.
5. Enter the following definition details for the new generic OneData matching rule:
a. In Name, provide a suitable name for the rule.
b. In Hint, enter a suitable description of how the score and pattern are configured.
c. Select Ignore Empty Fields to exclude fields without data for creation of the match score. If this field is not selected, then the empty fields are also considered for match score creation, according to the weightage specified.
d. In Definition, enter a suitable description for the rule.
e. Specify the following rule mapping details:
Field | Value |
Attribute | Enter a name for the attribute: |
Weightage | Enter a weightage value (in percentage) to be given to the attribute with respect to the actual match score. For example, consider that the Name, Street, and City attributes are given weightages of 5, 3, and 1, respectively. When the similarity score is calculated, Name is weighted 5 times, Street, 3 times, and City, 1 time in the final similarity score. |
Is Required | Select the check box only if the attribute meets one of these conditions:  Is an inbound attribute. Must have a Project-Object Mapping created. Note: If you select Is Required, you must define the Project-Object Mapping. In case there is no Project-Object Mapping for the attribute, validation errors occur during rule execution. |
Algorithm | Select a suitable matching algorithm from the available list of token-based, character-based, and hybrid algorithms. Cosine Coefficient Damerau-Levenshtein Dice Coefficient Jaro Jaro-Winkler Levenshtein Monge-Elkan Needleman-Wunsch OneData Similarity (recommended, and the default algorithm) Overlap Coefficient Sift3 Smith-Waterman Smith-Waterman-Gotoh |
f. If you require more than 10 attributes (provided by default) for the rule, click Add New Mapping and add the attribute details as described in previous step.
g. Click Save when you have added all the necessary rule definition and mapping details.
6. If the rule is a Locate Cleansing Threshold Mapping rule, select enter the following definition details:
a. In Name, provide a suitable name for the rule.
b. In Hint, enter a suitable description of how the score and pattern are configured.
c. Select Ignore Empty Fields to exclude fields without data for creation of the match score. If this field is not selected, then the empty fields are also considered for match score creation, according to the weightage specified.
d. In Definition, enter a suitable description for the rule.
e. Specify the following rule mapping details:
Field | Value |
Attribute | Enter a name for the attribute: |
Weightage | Enter a weightage value (in percentage) to be given to the attribute with respect to the actual match score. For example, consider that the Name, Street, and City attributes are given weightages of 5, 3, and 1, respectively. When the similarity score is calculated, Name is weighted 5 times, Street, 3 times, and City, 1 time in the final similarity score. |
Is Required | Select the check box only if the attribute meets one of these conditions: Is an inbound attribute. Must have a Project-Object Mapping created. Note: If you select Is Required, you must define the Project-Object Mapping. In case there is no Project-Object Mapping for the attribute, validation errors occur during rule execution. |
Algorithm | Select a suitable matching algorithm from the available list of token-based, character-based, and hybrid algorithms. Cosine Coefficient Damerau-Levenshtein Dice Coefficient Jaro Jaro-Winkler Levenshtein Monge-Elkan Needleman-Wunsch OneData Similarity (recommended, and the default algorithm) Overlap Coefficient Sift3 Smith-Waterman Smith-Waterman-Gotoh |
f. If you require more than 10 attributes (provided by default) for the rule, click Add New Mapping and add the attribute details as described in previous step.
g. Click Save when you have added all the necessary rule definition and mapping details.
7. In the Rules Management screen, select the exit criteria to process the project rules:
Evaluate all rules. To evaluate all rules in BOTH and MASTER patterns, so that
OneData returns the best match score.
The order of precedence for evaluation is first, BOTH pattern, and then MASTER pattern.
Note: Software AG recommends Evaluate all rules, as it evaluates all rules. However, it might affect performance as all rules are executed.
Exit on first good match. To evaluate rules in the priority set.
If a rule gives a score higher than the good match score, OneData stops rule evaluation without evaluating rules with lower priority. It then returns the score achieved and rule hint.
8. Select the Rule Selection check box to associate the rule to the project.
If you do not select Rule Selection, the rule is not associated with the project.
9. If you selected Exit on first good match as the exit criterion, in Priority, select the evaluation priority for the rule.
Note: The Priority must be unique for each rule. For example, do not set priority 3 for more than one rule.
10. Click Save Rule Assoication.
11. In Attribute and Attribute Type, define the required inbound attributes to match and outbound attributes if you require information on how OneData consolidates the consolidation records.
Note: Gold objects and the associated attribute are disabled when you select the outbound attribute type.
Attribute | Attribute Type | Description |
CLN_BusinessName | Inbound | |
CLN _StreetAddress | Inbound | |
CLN _State | Inbound | |
CLN _PostalCode | Inbound | |
CLN_BusinessName | Inbound | |
CLN _StreetAddress | Inbound | |
CLN _State | Inbound | |
CLN _Country | Inbound | |
WindowKey | Inbound | |
Hint | Outbound | A description on how the score and pattern were obtained. 400 characters is the preferred width of the staging column to which this attribute is mapped. To accommodate the possibility of a longer string, make suitable changes to the width of the staging column in the object definition so that the column holds the full string length. |
Pattern | Outbound | Using the customer domain example, the possible values: MASTER. Relevant only in a Multiple-Table Gold model. It indicates that only the customer attributes have matched. So there is a new address for the existing customer. BOTH. Relevant only in both Multiple-Table Gold and Single-Table models. In a Multiple-Table Gold model, it indicates that there is an address available. In a Single-Table model, BOTH is the only value possible as all customer information is available in a single table. 18 characters is the preferred width of the staging column to which this attribute is mapped. |
Score | Outbound | Values could be any number between 0 and 100. It indicates the similarity between the Staging and Gold records. Higher the score, better the match. 10 characters is the preferred width of the staging column to which this attribute is mapped. |
Match Type | Outbound | Type of match executed: Good Match. When the score is greater than or equal to the good match threshold. Suspect Match. When the score is less than the good match range, but greater than or equal to the Suspect Match threshold. Below Suspect Match. When the score is less than the suspect match range. No Match. When no match candidates are available. Manual Match. On manual execution of the match hook. 50 characters is the preferred width of the staging column to which this attribute is mapped. |
Matcher Action | Outbound | Includes the action set by the user in the Matching Configuration for each Match Type and details of how the action was executed. 200 characters is the preferred width of the staging column to which this attribute is mapped. Possible values are: Create Gold - Created Process best match – Linked to existing gold record Process only if single match – Queued for manual match Process only if single match – Linked to existing gold record Queue for manual matching - Queued Do not process - No action Manually linked to existing gold record |
Details | Outbound | Describes the reason for the matcher action. 200 characters is the preferred width of the staging column to which this attribute is mapped. Possible values are: Gold record created Single match candidate found Multiple match candidates found All match candidates below Suspect Match No match candidate found Left for manual match Manual matching executed Queued for manual match For example, let us assume you configured Action as Process only if single match for Good Match. Subsequently, the matcher executed the action Queued for manual match. The Details attribute explains that the reason for the matcher action. The reason might be, for example, Multiple match candidates found. |
Project | Outbound | Name of the data quality project used to run the matching process. 400 characters is the preferred width of the staging column to which this attribute is mapped. To accommodate the possibility of a longer string, make suitable changes to the width of the staging column in the object definition so that the column holds the full string length. |
Rule Name | Outbound | Name of the rule used in the matching process. 400 characters is the preferred width of the staging column to which this attribute is mapped. To accommodate the possibility of a longer string, make suitable changes to the width of the staging column in the object definition so that the column holds the full string length. |
Exit Criteria | Outbound | Describes how the rules were executed to obtain the score. The value provided here is the Exit Criteria value specified when defining rule associations for the project in the Rules Management screen. 50 characters is the preferred width of the staging column to which this attribute is mapped. Possible values are: Evaluate all rules Exit on first good match |
12. If an inbound attribute requires Project-Object Mapping, select Is Required.
Note: If you select Is Required, you must define the Project-Object Mapping. In case there is no Project-Object Mapping for the attribute, validation errors occur during rule execution.
13. Click Save Mapping.
14. Repeat the procedure from step 2 onwards until you have added all the inbound and outbound attributes necessary for the project.
16. Add rule mappings for the project.