Restricted access

December 3, 2010

External Data in Business Data Integration

Filed under: Data Integration — Tags: — Katherine Vasilega @ 7:59 am

One of the great opportunities for business data integration today is the potential for buying and sharing data via the Cloud. Adding the data originating outside of your organization can be a strategic advantage, as it provides a huge value increase of your data integration efforts.

The majority of organizations implement business intelligence systems that are internally focused. They show data from within the company: products, sales, contact information of customers, etc. These systems might be able to present you only the opportunities embedded in the data you already have.

But how much do you know about “good” and “bad” customers except for their names and addresses? Do you know their income level, likes and dislikes, social habits? This information can be crucial for your marketing policy. It gives you a ground for the right decision-making. It’s a good idea to plan your business data integration initiatives in a way that they will support data from external sources sold as commercial Web data services.

A data integration solution can include connecting with Web services that provide demographics and social information about your existing and potential customers. You can receive up-to-date information based on phone verification services, demographics services, social research studies, etc.

Income level, household size, number of cars owned, level of crime in the local area, and average level of local Facebook usage are all examples of external data that can be used to evaluate your customers. Tying this information to your existing customer records will give you a wider perspective of your target audience.

And the final remark: before subscribing to the external Web data services, you have to evaluate your data quality and the ability of your business data integration solution to handle large volumes of data.

April 5, 2010

ETL Faces New Challenges

Filed under: Data Integration, ETL — Tags: , , — Olga Belokurskaya @ 3:30 am

Today, well-established data integration process is necessary for a sound business. Business information is a valuable asset; companies’ decision-makers depend greatly on the data they receive, its quality, value, and actuality. As the amounts of data companies work with grow exponentially, the requirements to ETL systems get more complex. Today, ETL providers face some new challenges along with traditional data integration issues:

Scalability. ETL systems need to be able to process large volumes of data that intend to keep growing. Moreover, today’s business reality requires getting more data in less time. So, scalable ETL is a must.

Operability. A large company’s IT system comprises multiple disparate sources of business-critical data, such as databases, CRM systems, etc. These days, ETL tool should have connectivity to all those systems. Ah, moreover, data integration between all data sources often requires complex transformations to make the data fit the formats common for this or that system.

Real-time data integration. This requirement is being heard more and more often. The need for real-time data demands from ETL systems the ability to process extract-transform-load operations and gather all the data in a standard, homogeneous environment in a really short period of time.

Finally, the Cloud. As cloud offerings get mature and provide some beneficial solutions (especially for small and mid-sized business), companies choose to move parts of their applications to the cloud. Providing the connectivity to cloud systems is a today’s ETL challenge, as well.

March 5, 2010

Database Integration: On the Importance of Data Quality Standards

It’s a sad fact, but many organizations realize the poor quality of the data in their databases, only when it comes to database integration. Data quality issues are among the common reasons for data integration failure.

This neglecting attitude to data quality lies in the fact that companies often don’t understand how much data quality impacts business processes. Thus, each data source or database a company uses may have its own rules and standards for data quality. The issues, however, evolve as soon as the database integration started in order to get a unified look at, for example, company’s customers’ data.

Those issues may come out of the difference of data fields, for example, or data formats, so the same contact may be represented differently in different databases. Thus when it comes to database integration, it can’t be performed correctly due to those differences, which may lead to data duplication, and many more data quality issues. In fact, in the result of integrating several databases of poor quality, a company gets one big database of poor quality. This means that database integration was in vain, as it failed to achieve its main goal of providing the company with a general view of business data, while the integration expenses were significant.

Unfortunately, data quality technology does not always allow organizations to fix poor data. So, it’s much wiser to implement company-wide standards for data quality to prevent the appearance of data quality issues associated with integration of data from heterogeneous sources, then to perform data cleansing and other data quality procedures afterward.

March 3, 2010

Data Integration Is Not About Tools, It’s About Strategy

Filed under: Data Integration, Data Migration, ETL — Tags: , — Olga Belokurskaya @ 4:13 am

Today, organizations face increasing data integration challenges. The amounts of data grow progressively, demanding for new levels of data protection, and making data migration even more complex.

At the same time, business demands access to a real-time information and quality data to make right business decisions. There are plenty of technologies that are able to address data integration challenges, though some of them get old-fashioned, some continue to mature, etc. Thus according to Forrester Research, MDM and data quality services continue to mature, while ETL (extract, transform, and load) and data replication “have reached the Equilibrium phase,” and some technologies are moving to a decline.

But that doesn’t mean that a technology or tools which are in the top of the list today, may become a successful solution for data integration challenges. Lots of organizations regard data integration as mostly a technological process, not taking into account how it impacts organization’s long-term plans and the success (or failure) of business. However, successful data integration is mostly about strategy. And when a strategy is defined, the choice of tools get’s much easier, and there’s no risk that a chosen technology won’t cope with the task.

February 18, 2010

Data Migration: Challenges of Moving Data to a CRM

Filed under: Data Integration, Data Migration, Data Quality — Tags: , — Olga Belokurskaya @ 4:36 am

CRM is a great solution to effectively manage company’s customer data. However, to ensure efficiency, get the most out of CRM system, and avoid CRM failure, special attention should by paid to the data that is being migrated to a CRM, and how it is being migrated. There are challenges that may affect the process of data migration:

  • Migration from heterogeneous sources. Data migration from various sources to a CRM is a challenge, so it should be approached with care and strategy. Migrating the data just as is leads to a CRM failure, as you simply take siloed data and move it to another system (the data still remains siloed). So, before data migration takes place, there is a work that should be done to ensure the quality of the data that is targeted to a CRM.
  • Data mapping. To migrate data, one should map the source and the target. That’s clear. The problem is, that in different sources data fields possessing equal information, may be named differently (for example, “Username” and “Account”), or vice versa, equally named fields may stand for different values (“Name” may stand for a first name solely, or for the full name, including first name and surname). That’s why it’s very important to map fields in a data source with appropriate fields in a CRM application.

It’s highly probable that before data migration is started, data will need to be transformed, in order to make it appropriate for a new CRM system. Surely, it should be cleaned from duplications, incomplete records, outdated records, etc. Only when data is prepared, clean and in the appropriate format it may be migrated to a CRM.

February 16, 2010

When Developing Systems Architecture Think About Data Integration

Filed under: Data Integration, Data Quality — Tags: , — Olga Belokurskaya @ 6:34 am

“The data, and integration strategies around the data, is something that most figure is there, will be there, and requires very little thinking and planning.” – This is what I’ve read today at ebizQ.

This again supports the idea that data, though being “the biggest companies’ value”, is still often being neglected. And this results in data integration and quality issues, providing inconsistent data and ruining the entire idea of data integration as a way to provide a clear view on enterprise data. The data that is important for business decisions.

In fact, very often when it comes to designing and developing the architecture, all the attention is focused on technical side of the process. Thus data integration strategies become an afterthought making it difficult to meet business requirements. So, the message is that provisions for data integration should be made at the level of the development of enterprise systems architecture.

February 12, 2010

Data Integration and Data Quality Require Consolidation

Filed under: Data Integration, Data Quality — Tags: , — Olga Belokurskaya @ 10:08 pm

Lately, there have been talks about the need for a union between data integration and data quality solutions, as no integration initiative can be called successful without the quality of the data provided.

Today, the functionality that provides data quality becomes a requirement for data integration tools. However, data quality still remains a challenge. According to Gartner, only 25% of organizations will proactively and comprehensively include data quality processes and competencies in their data integration work in 2010. This might mean that a large number of enterprises will have to struggle with incomplete, inconsistent and inaccurate data, as their main focus is on the mechanics of data integration process with no connection to data quality.

Technology providers apply more efforts to combine data integration and data quality processes into one data management process. Steps have already been made toward the consolidation of technology offerings, instead of using distinct data integration and data quality solutions. The benefits of this consolidation are quite obvious – a consistent and complete view of the business data, and as a result, the ability to change business processes and models faster to gain the market.