Data Quality allows data migration and data integration to flow

Data migration and data integration – a bad hand with poor data quality

There are countless scenarios for customer data but also for all data stored in databases in which poor data quality obstructs data migration and integration. In contrast to incorrect addresses, there are no electronically accessible references or models in other data domains, such as product or material master data, on whose basis discrepancies can be automatically detected and cleansed. Here the requisite measures have to be identified and specific transfer rules defined on a case-by-case basis before migration or integration.

Data is not available

If data is required for the destination system but has not been created in the source system, problems in the data migration are the inevitable result. Let's take the example of the transfer of a support database to the new CRM system: if a courtesy title is a stated obligation in the new system (e.g. for personalized e-mails or letters), but this was not mandatory in the old system, the instances without a courtesy title cannot be adequately classified in the destination system.

Data is outside the requisite value range

Let's stay with the support example: if the courtesy title of the contact person in the destination system is based on clearly defined values which differ from those in the source system and for which there is no conversion (transformation), they cannot be transferred. In this respect, it makes no difference whether the errors were caused by inconsistent application by the personnel or whether the rules for the courtesy title were altered at some stage but not documented.

Data is not available in the requisite format

If the data is not available in the source system in the requisite format for the destination system, this could mean that the data cannot be transferred, or it will cause problems during subsequent use and supply incorrect results.

Data contains orphan data records

Each data object in a database is always related to a higher-ranking object, e.g. a contact is always assigned to a company or an item to a quotation. If these assignments are missing in the source system ("orphan data") but are a mandatory requirement for the data model of the destination system, errors will occur during data transfer. This is especially problematic when merging data from different systems e.g. ERP and CRM systems with different database schemes.

Customer data with incorrect or out-of-date addresses

A migration is often the starting signal for a comprehensive analysis and cleansing of incorrect or out-of-date data. Moreover, if certain data is to be linked together in a data integration, appropriate measures to significantly increase the data quality can be taken before transfer.

Customer data with different address formats

Addresses with different formats mainly occur when data from several source systems is to be merged, but the data structures were variably defined, e.g. if the street and house number were entered in one field in the source system, but separate fields are provided in the destination system. Or if certain attributes for international addresses were written in fields not provided for this in the old system, because the requisite fields were not available.

Duplicates in customer and other data

Duplicates in any form of data can falsify evaluations and cause unnecessary work and costs on account of multi-processing. For example, all the data of a contact is recorded as a separate entity in a simple address management system, which means that there are a large number of redundant data records for a company with several contact persons. These data records must be separated and customized to the structure of the destination system for transfer to the new CRM system.

Linking data from different systems

Objects are often stored with different names in different systems, so that automatic merging through comparing character strings is inadequate. This can be caused by different standards for the storage of master data or by different data structures in the source and destination system. For instance, if contacts are stored in the hierarchy Company and Contact Person in the database of the sales department, and Company and Contact Person are stored as a unit in a record in the support database, the support data must be reclassified during transfer to a CRM system and merged with that of the sales database.

Uniserv Data Quality Solutions for data migration and data integration - analysis, monitoring, cleansing

The modular Data Quality Solutions from Uniserv help you to transfer your data to other systems in your IT infrastructure without any problems. You maintain the momentum of your operative activities and can make productive use of your new applications more quickly thanks to optimized data quality. And you are able to meet the time and budgetary requirements of your data migration or data integration project to pinpoint accuracy.

Data Quality Explorer for data migration: The Explorer enables you to determine the actual state of your data before transfer, especially if the documentation of the old systems is inadequate. You can derive the transformation rules for the actual migration process from this.

Data Quality Monitor for data migration: This enables you to monitor and count the breaches of specified business rules (off-line). Above all, this makes good sense if incorrect data has to be cleansed in the old system but an appropriate check is not integrated there.

Data Quality Batch Suite for data migration: The complete product suite for batch transfer from different data sources, transformation of record structures and field contents, validation and cleansing of data as well as flexible provision of the data for the new systems in any desired format.

Our contribution to smooth data migration: Profiling of the data sources of the old system and the monitoring of threshold values for specified business rules. Transfer of the data from different data sources, transformation of record structures and field contents, automatic duplicate recognition and clustering of name and address data. The data validation, enhancing and consolidation functions supplement the data quality tool box for data migration. Uniserv DQ Explorer, DQ Monitor and DQ Batch Suite - our contribution to your successful data migration project.

Amongst other benefits, all the Uniserv Data Quality Solutions for data migration and data integration enable you to profit from the following:

• Extensive analysis of the existing data and data structures (data profiling)
• Quick transformation of record structures and field contents
• Monitoring of the data quality in sources and destination systems at initial data creation and for changes (Data Quality Monitoring)
• Reliable cleansing of data errors, duplicates as well as impermissible field entries (data cleansing) according to defined rules and by matching against reference data
• Faster use of higher performance systems through the smooth transfer and merging of data from different systems with the highest quality