Gartner has lately highlighted a number of technology trends CRM professionals should pay attention to as they plan their CRM architecture.  Social CRM initiatives are among those trends. Gartner predicts spending on such social CRM initiatives as customer communities and social media monitoring to grow up to 10% of spending in 2020.

What will this mean for traditional vendors? They will have to implement social capabilities into operational CRM. By social capabilities meant three areas: internal collaboration tools, internal communities hosted by the business, and outside social networks monitoring.  However, there is a resistance to social CRM initiatives coming from legal departments whose main concern is the level of corporate governance.

Among another trends that will impact CRM Gartner also named:

Turning to cloud computing to decrease operational costs;

Corporate performance management (CPM) including includes data-driven planning, budgeting and forecasting activities often includes sales and marketing analytics;

Web-oriented architectures which are not yet applicable in a majority of cases in complex enterprise applications, but will be soon, according to experts.

Careful analysis of the quality of data in your current system should be done prior to migration. Good data can save quite a bit of time (and budget) during migration. In fact, this analysis and replacement set up during the migration will help you cleanse and improve your data. Below are some points to pay attention to.

- There fields that require being filled in for records to be migrated, such as company names for accounts, customer names for contacts, and so on. The main issue here is some of the fields missing information.

To cope with the issue, think what values should be entered into these empty fields. The key factor here is to make it convenient for you to use the data when it is migrated.

- It is important to ensure that values entered in the fields of one system meet the data type requirements of the system where the data is migrated, and there is no conflict during the migration.

- One more issue is data logical duplicates that are presented in different ways in the field. The thing is that there may be different ways of naming the same value. So you need to replace all the different names with one. If there’s no possibility for automatic replacement in your current system, you’ll need to add these replacements within the data transformation to be built for the project.

Learn more about the data quality issues that may occur during data migration from the whitepaper on “The Three Most Common Data Integration Problems”

Providing centralized view of related data originating from different systems is one of major benefits of data integration. More and more enterprises implement data integration initiatives to govern their data more productively. Thus they spend time on analyzing the market of data integration tools in search of solutions that would fit their peculiar needs, and so on.

Well the importance of having centralized view of data is obvious, but there are several approaches of getting the data centralized.

According to David Jennaway, CRM and data integration expert, two main approaches are

- Using reporting mechanisms, and combining reports to analyze data from different sources, but leaving the data in the sources systems.
- Copying data from different systems into one system, providing this system with all the needed data.

Each approach has its advantages.

Reporting option is easier and cheaper for implementing, and designing a report is much simpler than creating a robust process to copy data. Moreover, the reported data is not likely to be backdated, as it comes directly from the data source, thus avoiding the latency which most data duplication processes have within them.

Copying data into one system brings flexibility for having needed data gathered in one system eases searching and displaying required items. It is also possible to edit and make changes to the data outside the systems it originates from, though it will require well-established data synchronization processes.

While reporting may seem a simpler approach in the start, data integration turns much more beneficial, easing working with enterprise data, that may come from multiple sources from multiple departments (no argue, in this case analyzing multiple reports may be quite time consuming).

As your business grows, requirements for your data storage change. At the same time, the IT market offers new, diverse software to catch up with or even get ahead of your needs. At a certain stage, you face the necessity of switching to a new system. What are the hidden problems related to data migration (DM)? How can you omit them and get the most of your DM initiative?

Studying the new system’s capabilities is an important step not just in preparing requirements for your data migration/integration project, but in selecting the system in the first place. Here are some things to do to ensure your choice is right, and you understand the system:

• Make sure the system really does meet your requirements and will be able to satisfy your needs.
• Make sure the components and functions/capabilities of the system are convenient for you to work with and perform the actions you need to perform with your data.
• Analyze how the new system communicates with other systems. A good import/export mechanism included in the new system is a useful feature, but it’s not always sufficient. So, it is important to study the system’s API or SOAP capabilities in order to define whether the system you’ve chosen provides a full-scale toolset for data integration or additional alternative integration methods will be needed, such as direct database access, if it is enabled by the system. This issue will definitely make the project more labor-intensive, time-consuming, and, as a result, more costly.

Learn more about the challenges of data migration in our white paper “The Three Most Common Problems Faced During Data Migration”

Real-time data integration is a real buzzword today, and an object discussed more and more actively.

What differentiates real-time approach from traditional data integration is the character of the process. While traditionally the process of data integration supposes moving large blocks of data incrementally over a 24-hour period, real-time approach means moving data continuously (though sometimes there is a few seconds of latency).  Assuming that traditional data integration may take significant amount of time, or face challenges like window of tome expiration before an operation is completed, the idea of constant real-time integration of smaller portions of data seems more efficient as an approach.

Companies look at real-time data integration as at the way of cutting costs and increase revenue from IT initiatives. Real-time data integration tools implementation provides things like following:

Immediate access to up-to-date business information
Confidence that decision making is based on latest data
Possibility to see any changes about customers as soon as they occur, etc.

Thus, real-time data integration keeps gaining popularity as more efficient way of getting the right business information in the right time. Still the choice and implementation of real-time data integration solutions supposes a significant work to be done.

And again about data quality. According to Gartner, though the adoption of data quality tools increased, organizations still lose more than $8 million annually due to poor data quality. Moreover, achieving comprehensive data quality processes is something most organizations are very far from. The reason is that mainly IT staff utilizes data tools, because they are complex and difficult to understand for non-IT users, resulting in their slow adoption by most of the supposed users.

Gartner provided some recommendations called to help organizations improve data quality:

First is for vendors to make data quality tools simpler to use so that not only IT people, but business responsible could use them and start being more accurate with their data in terms of quality.

“In particular, providing data profiling and visualization functionality (reporting and dashboarding of data quality metrics and exceptions) to a broader set of business users would increase awareness of data quality issues and facilitate data stewardship activities,” Ted Friedman, an analyst, mentioned in his report.

The advice for organizations is to consider pervasive data quality controls throughout organization’s infrastructure and investments in technologies applying data quality rules to data in terms of capturing and maintenance, as well as downstream.

What are connectors for data integration tools? Allowing to minimize or even avoid hand-coding required to move data out/into different data sources, they have become a significant part of data integration solutions. However, it’s not right to think, that simply having a bunch of connectors is always enough for easy and smooth data integration. Well, apart from providing connectivity, there may be some peculiar requirements to connectors. They are, according to John Bennett’s, a marketing consultant and business writer, whose article I’ve read lately:

• Different level of security and access requirements for data sources. – In other words, integrating protected data, you need to ensure that it remains protected after the integration. This is especially actual for the data in the cloud.
• The necessity of transformation. – Very often, data requires to be transformed somehow before being integrated in some application or another data source. This may be some data formats conversion or unification, etc.
• Being a part of an integration solution that supports access controls, transformation, auditability, etc. – This means, ensuring that all operations over the data are complete, and data is up-to-date, before integrating it.

Well, what’s all this about? It’s obvious, that very often data needs a lot more than to be integrated as is. So searching for a data integration tool or solution, make sure its connectors can cope with the operations your data may require. Otherwise, you’ll still need lots of hand-coding which is not a pleasant (and cheap) thing, and there are risks of mistakes and bugs leading to data integration nightmares – the things you definitely wish to avoid. =)

When choosing a data integration solution, it is important to keep in mind a lot of things, the most crucial being that while many companies tend to underestimate the TCO (comprising license fee, hardware costs, and labor costs) of data integration, for every dollar spent on integration software, enterprises spend $6 on subsequent implementation and support. This goes for proprietary software.

Under tough economic conditions for many companies the total cost of ownership of enterprise of proprietary data integration solutions is becoming prohibitive. One may start to think of data integration as of something that constantly consumes enormous resources, both human and financial, and that would make a point. But another point is that to reduce the total cost of ownership, you have to change the cost structure, that’s it:

• Today that there is a number of open source solutions available for evaluation and real-world projects at no charge. With open source, you can practically eliminate licensing fees at all.
• As far as hardware costs are concerned, eliminating the vendors whose solutions do not provide the acceptable level of openness can result in a considerable saving.
• When it comes to labor cost, there are two main criteria.
  • The first of them is whether the solution is straightforward enough for a non-trained user to use, and whether it is effective, which means that the user is able to do the job in minimal time.
  • The second criterion is the flexibility of the product, the extent of reusability of its configuration for follow-on tasks.

  Commonly, open source manages to provide both, oriented on and supported by a large community that aims to consistently enhance the development.

Learn more about how to reduce the total cost of ownership from our “Guide to Reducing Data Integration Costs.”

Business intelligence can’t be imagined without data integration and extraction. Choosing and implementing the right technology is a key to successful business intelligence deployment. However, many organizations in pursuit of the best of the best of the best extract, transform, and load (ETL) product overlook the fact that ETL technology should primarily fit organization’s business needs.

Another mistake organizations often make is expecting that ETL implementation will help finishing the project quicker. But they do not take into account the time needed to find and implement the technology, to train personnel to use the tool and be productive with it. According to TDWI, this period may take up to six month.

Expecting for immediate savings is not the right approach either. In fact, the investment in high-end technology acquisition is, normally, equal to the total labor cost for an average data warehouse built with hand-coded ETL. Adding to this the development and annual support cost for a high-end ETL tool, we receive three-four years to get the point of make-out.

Considering all those factors will ensure that the stakeholders’ expectations of a project are appropriate and ROI analysis is right.

Being a type of operational data integration, data synchronization stays a bit apart from other operational data integration types, such as data consolidations, collocations, migrations, and upgrades.

They differ in terms of the main mechanism of dealing with data in databases.  While migration, consolidation, etc. suppose moving databases (either several databases to a single one, or a database from one system to another), synchronization suggests moving, exchanging data between different databases. The pluses of synchronization include:

• Leaving database investments and the business processes that depend on them intact
• Being extremely helpful when managing related databases which are impossible to consolidate (let’s take, for example Salesforce and QuickBooks synchronization).

However, the necessity to have a permanent integration infrastructure for daily data feeds (as it is mentioned in TDWI research), which is costly both in term of initial investments and maintenance, is the main drawback of synchronization.

In terms of tools, ETL is the most popular choice for database synchronization.