Restricted access

March 5, 2010

Database Integration: On the Importance of Data Quality Standards

It’s a sad fact, but many organizations realize the poor quality of the data in their databases, only when it comes to database integration. Data quality issues are among the common reasons for data integration failure.

This neglecting attitude to data quality lies in the fact that companies often don’t understand how much data quality impacts business processes. Thus, each data source or database a company uses may have its own rules and standards for data quality. The issues, however, evolve as soon as the database integration started in order to get a unified look at, for example, company’s customers’ data.

Those issues may come out of the difference of data fields, for example, or data formats, so the same contact may be represented differently in different databases. Thus when it comes to database integration, it can’t be performed correctly due to those differences, which may lead to data duplication, and many more data quality issues. In fact, in the result of integrating several databases of poor quality, a company gets one big database of poor quality. This means that database integration was in vain, as it failed to achieve its main goal of providing the company with a general view of business data, while the integration expenses were significant.

Unfortunately, data quality technology does not always allow organizations to fix poor data. So, it’s much wiser to implement company-wide standards for data quality to prevent the appearance of data quality issues associated with integration of data from heterogeneous sources, then to perform data cleansing and other data quality procedures afterward.

February 18, 2010

Data Migration: Challenges of Moving Data to a CRM

Filed under: Data Integration, Data Migration, Data Quality — Tags: , — Olga Belokurskaya @ 4:36 am

CRM is a great solution to effectively manage company’s customer data. However, to ensure efficiency, get the most out of CRM system, and avoid CRM failure, special attention should by paid to the data that is being migrated to a CRM, and how it is being migrated. There are challenges that may affect the process of data migration:

  • Migration from heterogeneous sources. Data migration from various sources to a CRM is a challenge, so it should be approached with care and strategy. Migrating the data just as is leads to a CRM failure, as you simply take siloed data and move it to another system (the data still remains siloed). So, before data migration takes place, there is a work that should be done to ensure the quality of the data that is targeted to a CRM.
  • Data mapping. To migrate data, one should map the source and the target. That’s clear. The problem is, that in different sources data fields possessing equal information, may be named differently (for example, “Username” and “Account”), or vice versa, equally named fields may stand for different values (“Name” may stand for a first name solely, or for the full name, including first name and surname). That’s why it’s very important to map fields in a data source with appropriate fields in a CRM application.

It’s highly probable that before data migration is started, data will need to be transformed, in order to make it appropriate for a new CRM system. Surely, it should be cleaned from duplications, incomplete records, outdated records, etc. Only when data is prepared, clean and in the appropriate format it may be migrated to a CRM.

February 17, 2010

ETL Tools: How to Make a Choice

Filed under: Data Integration, ETL — Tags: , — Olga Belokurskaya @ 5:55 am

There is a wide variety of ETL tools available on the market, starting with solutions with minimum functionality and ending with tools that help solve complex tasks. There is also a choice between proprietary offerings and open source ETL tools, Web based and desktop solutions. Selecting an ETL tool requires some effort. When choosing an ETL tool for a particular company, a lot of things should be taken into consideration, including currently used data management processes, technologies utilized, IT staff available, etc.

Thus, to evaluate ETL tools and make a decision in favor of a certain offering, a set of questions should be answered, such as:

  • The operating systems supported by an ETL tool,
  • The volume of data the tool is able to handle in a given period of time,
  • Data sources and data formats the ETL tool supports, etc.
  • There is also significant to find out on what conditions maintenance and support are provided (paid, free of charge, etc.)

Besides, company’s requirements to data integration should be analyzed, and compared with the functionality that different ETL tools provide before making a decision and purchase a tool. Thus, a company may avoid paying for the functionality they are not going to use.

February 12, 2010

Data Integration and Data Quality Require Consolidation

Filed under: Data Integration, Data Quality — Tags: , — Olga Belokurskaya @ 10:08 pm

Lately, there have been talks about the need for a union between data integration and data quality solutions, as no integration initiative can be called successful without the quality of the data provided.

Today, the functionality that provides data quality becomes a requirement for data integration tools. However, data quality still remains a challenge. According to Gartner, only 25% of organizations will proactively and comprehensively include data quality processes and competencies in their data integration work in 2010. This might mean that a large number of enterprises will have to struggle with incomplete, inconsistent and inaccurate data, as their main focus is on the mechanics of data integration process with no connection to data quality.

Technology providers apply more efforts to combine data integration and data quality processes into one data management process. Steps have already been made toward the consolidation of technology offerings, instead of using distinct data integration and data quality solutions. The benefits of this consolidation are quite obvious – a consistent and complete view of the business data, and as a result, the ability to change business processes and models faster to gain the market.