The Batch Cleansing and Matching Processes
Faster processing is possible by concurrently processing records in batches. The batch cleansing and matching are background tasks available in the Project-Object Mapping screen for the Cleansing and Matching projects.
The batch processes require a job. When you create a project-object mapping, OneData a default batch job for both cleansing and matching. If, for some reason, either the default cleansing or matching batch job is not available, OneData creates one when you execute the batch job for the first time. You cannot modify or delete default batch cleansing and matching jobs because OneData uses them to track manually triggered jobs in the Job Center. You can also schedule batch jobs using the Add New Job option in the Batch <Cleansing or Matching> Execution screen.
Advanced logging options are also available.
You can schedule data quality cleansing and data quality matching jobs through a Job Scheduler.
How Does the Batch Cleansing Process Work?
The batch cleansing process performs the following operations:
1. It connects to the configured data quality cleansing server (for example, webMethods Locate).
2. Cleanses records that are in the Unassigned state in the Consolidation (Staging) table.
3. When the cleansing process is completed successfully, moves the records to the Cleansed or Cleansed (Pending Review) status.
How Does the Batch Matching Process Work?
The batch matching process performs the following operations:
1. It connects with the configured data quality matching server to execute the data quality matching process which detects duplicate data.
2. Depending on the result from the data quality matching process, the batch matching process creates a new or updates an existing Gold record with the attributes from the record in the Consolidation object based on survivorship rules.
3. Links the Gold record to the record in the Consolidation object.
4. After processing records successfully, the batch process changes the status of the records from Cleansed or Cleansed (Reviewed) to Create_Auto, Linked_Auto, or Queued for Manual Match.