Restricted access

April 29, 2011

Data Integration Challenges: Define Your Customer

Filed under: Data Integration — Katherine Vasilega @ 4:56 am

The IT and business alignment is a widely discussed challenge of data integration. The major data integration problem adds up to this: define customer.

Data from different functional areas doesn’t join up: sales orders are associated with the newly contracted customers, but the marketing campaign data is associated with prospects. Is a customer someone who’s actually bought something from you, or is a customer someone who’s interested in buying something from you? Should a definition include a certain demographic factor that reflects your typical buyer? If sales, marketing, service, and finance can all agree on a single definition of customer, then all the associated transactions could be easily integrated.

The thing is that all these specialists have their understanding of the word “customer”. That is why it is next to impossible for them to agree on a single definition and you have to somehow manage data integration without it.

To solve this issue, you can define what each functional area (and each CRM system) means by “customer”. This is how we know that customer data coming from a marketing system includes prospects, as well as existing customers. With this information, you can build a semantic model to understand how the different definitions of customer relate to one another.

Using this model, it would be possible to associate supply data with parts, cost data with product class, marketing data with brands, and so on. The relationships among these entities allow for data integration from different functional areas. This semantic model may be complex, but try to accept it and don’t head for simplifying it. The world is complex. Data integration requires a sophisticated understanding of your business, and standardizing vocabulary is not going to be the right answer to this challenge.

April 20, 2011

iPaaS: A New Trend In Data Integration?

Filed under: Data Integration — Katherine Vasilega @ 6:51 am

iPaaS (integration platform-as-a-service) is a development platform for building integration applications. It provides a set of capabilities for data integration and application integration in the Cloud and on-premises.

There are very few vendors offering iPaaS solutions at the moment. Although Gartner recognizes and uses the term, it still sounds confusing to researchers and data integration experts. So how does iPaaS work and can it benefit your data integration efforts?

Integration platform delivers a combination of data integration, governance, security and other capabilities to link applications, SOA services, and Cloud services. In addition to basic features that a Cloud solution should have, such as multi-tenancy, elasticity, and reliability, there are other capabilities relevant for iPaaS:

    1. Intermediation, the ability to integrate applications and services using the Cloud scenarios, which include SaaS and Cloud services, on-premises apps and resources.
    2. Orchestration between services, which requires connectivity and the ability to map data.
    3. Service containers to enable users publish their own services using either RESTful or SOAP technologies.
    4. Security covers the ability to authenticate and authorize access to any resource on the platform, as well as to manage this access.
    5. Enterprise Data Gateway installed on-premises and used as a proxy to access enterprise resources.

Data integration and application integration with and within the Cloud is the concept that business owners should consider nowadays. As of today, iPaaS would mostly appeal to companies that don’t mind building their own IT solutions or to ISVs that need to integrate Cloud silos they have created previously. It will be interesting to see whether iPaaS will become the next trend in the data integration discipline.

April 2, 2011

Is Your Data Integration Technology Outdated?

Filed under: Data Integration — Katherine Vasilega @ 7:49 am

Spring is a good time to get rid of the old stuff and check out something new. This might as well be the time to upgrade your data integration tools. How can you learn if your data integration solution is outdated and should be replaced by something more productive? May be it just needs a little tuning? Here are the main check points to see if your solution’s performance still fits the industry standards.

Data transformation schemas
deal with both data structure and content. If data mappings are not as well-organized as possible, then a single transformation may take twice as long. Mapping problems can cause small delays that add up. The solution to the transformation issue is to make sure that data maps are written as efficiently as possible. You can compare your data integration solution to the similar ones to understand if the data transformation runs with the required speed.

Business rules processing are specific rules for the data that has to be validated. Too many rules can suspend your data integration processes. You have to make sure that the amount of rules in you data integration system is optimal, meaning that there are not too many of them running at the same time.

Network bandwidth and traffic—in many cases the performance is hindered not by the data integration tool itself, but by the size of the network you use. To avoid this issue, you need calculate the predicted performance under various loads and make sure you use the fastest network available for the data integration needs.

Data integration solution reminds a car: it can run but become slow if it is not properly tuned and taken care of. As we become more dependent upon the data integration technology, our ability to understand and optimize the performance issues will make a substantial difference.

March 10, 2011

The Key Data Integration Strategies for Successful CRM

Filed under: Data Integration — Katherine Vasilega @ 6:39 am

One of the great values data integration provides is a possibility to gain a deeper insight into one’s customers. It is not surprising that data integration with CRM (customer relations management) systems is one of the main directions in the industry development. As more companies choose managing customers electronically, it is quite useful to apply the most effective data integration strategies to pay-off for CRM investments.

The recent survey by the data integration experts and authors—Christopher Barko, Ashfaaq Moosa, and Hamid Nemati, —explores the significant role of data integration in electronic customer relationship management (e-CRM) analytics. They reviewed 115 organizations including both B2B and B2C companies and sorted out four data integration initiatives that provide for better CRM:

    1. Integrating more data sources. The research shows that the total value of CRM project increases when you integrate more data sources. As sales people are using more channels than ever before to reach prospects and customers, no wonder that data integrated from all these channels is more efficient, than when stored in the isolated silos.

    2. Integrating offline data with online data gives a better picture of customer’s buying habits. 62 percent of respondents said they integrated these data sources, while 30 percent did not. Not surprisingly, those who integrated the online and offline data experienced greater value from their e-CRM projects.

    3. Integrating external data (e.g., from social media sites) into the central repository. 74 percent integrated external data in some form, while 26 percent did not. The companies that practice external data integration in their e-CRM projects enjoy significantly more benefits.

    4. Using a centralized data warehouse or a CRM-specific data repository does provide a deeper customer insight. Those who used a decentralized data repository (legacy databases, operational data stores) experienced significantly less benefits than those who centralized their data storage.

As the number of marketing channels used to communicate with customers continues to multiply, so does the number of places used to store the data. The research reveals that the most efficient data integration strategies include integrating different kinds of data from multiple channels and keeping it in the central repository. These data integration best practices help ensure marketing efforts have a positive effect on sales.

March 5, 2011

How Can Data Governance Serve Data Integration Projects?

Filed under: Data Integration, Data Quality — Katherine Vasilega @ 3:56 am

Data governance initiatives in an organization are intended to cover data quality, data management, and data policy issues. These activities are carried out by data stewards and a team that develops and implements business rules for administrating the use of data.

The focus on data governance is essential when the company has to implement a successful data integration strategy and use it for analysis, reporting, and decision-making. Here are some ways of making data integration projects more efficient with data governance:

    • It brings IT and business teams together. Data governance identifies what is really important to the business and helps establish business rules that are crucial for data integration.

    • A data governance program can help your company define and measure the potential ROI you get from maintaining data. You can use this information to calculate the ROI for data integration projects.

    • It helps you learn who’s responsible for the data quality. Data governance provides valuable information that enables to appoint data stewards and decision makers for data integration projects. Since data governance tells you who’s responsible for the data, you know where to go to resolve data quality issues.

    • Data governance can save you money, because it helps establish best practices and select cost-effective data integration and data quality tools.

Data governance and data integration are tightly connected with each other. You are not likely to enjoy data integration benefits without a strong governance program. On the other hand, data governance is only possible if your data is stored in an integrated system. My advice: make sensible use of both.

February 17, 2011

Data Integration in SharePoint 2010

Filed under: Data Integration — Katherine Vasilega @ 6:23 am

A survey by AIIM (Association for Information and Image Management) states that although SharePoint is being rapidly adopted by organizations, at least half of the companies that are implementing the platform don’t have business use in mind.

This might be a reason we don’t see millions of companies shifting their data integration initiatives into SharePoint. It may be only a question of time, as SharePoint 2010 comes with rich integration capabilities. Here are some of the features that can be leveraged for external data integration and application integration:

    1. Business Connectivity Services (BSC) is a new feature of the SharePoint platform that provides new means for external data integration into SharePoint 2010. It enables to create connections to external data sources through the use of SharePoint Designer or more complex scenarios with custom code development.

    2. Web Services can be leveraged by both SharePoint and external systems for data integration and application integration purposes. Common services include the ability to authenticate, search, and manage content. SharePoint 2010 also includes built-in RESTful Web services, which allows the integration of remote systems.

    3. Client Object Models are used to integrate SharePoint and other systems to provide a better usability. SharePoint 2010 introduces three new client API’s: ECMAScript Client, SilverLight Client, and .NET Managed Client. These object models enable users to access both SharePoint and other data sources from a single interface that does not have to be or look like the SharePoint interface.

    4. The CMIS (Content Management Interoperability Services) connector for SharePoint 2010 enables to perform content management functions between systems that comply with the CMIS specification.

There are many ways in which organizations can leverage SharePoint for their data integration needs. Nevertheless, the question on whether companies will start data migration and data integration into SharePoint 2010 in the nearest future remains open.

January 3, 2011

The Additional Value of Data Integration

Filed under: Data Integration — Katherine Vasilega @ 2:50 am

The data integration discipline has undergone an impressive expansion over the last decade. Up-to-date data integration practices influence various business processes across organizations. I have talked a lot about the value of data integration for decision-making, sales, and customer service. What else can data integration add to the way the business operates? Here are some thoughts on the key issue.

Green Technology

Recent climate changes and the rising cost of electricity have led many people to reconsider the environmental sustainability of data centers. Data centers today are reducing power consumption and server rooms by consolidating redundant data and virtualizing hardware servers. Data integration tools and techniques help these consolidations and make IT more ecologically friendly.

The data integration technologies can reduce the number of redundant databases, thereby cutting down on the number of servers, plus the budgets and resources they consume. Furthermore, data integration techniques, such as data federation and data services, can assemble data sets on the fly, without generating permanent databases that burn up server resources.

Collaboration and Education

Collaboration between the IT and business teams is a must for data integration initiatives. The collaboration among the technical and business people involved in data integration projects helps them learn more about the specific tasks of each party involved in these projects, as well as understand the value business and IT teams bring to the organization.

The best practices listed above help organizations achieve more far-reaching uses of data integration tools and techniques. Getting beyond the standard definition of data integration value brings a wider perspective to the business development.

December 30, 2010

Data Services in Data Integration

Filed under: Data Integration — Katherine Vasilega @ 3:51 am

Many data management professionals agree that the increasing volume and complexity of data require a new approach to data integration with its traditional tools and methods. Organizations today are expanding their usage of information-as-a-service (also called data services) to support new business requirements, such as real-time data integration, creating data domains, improving security, integration with unstructured data and external sources.

Let’s take closer look at a data services approach in the data integration. First of all, it helps bring in data from more sources. This provides a better presentation of the important information for decision-making, bringing a return on investment in a shorter period time, compared to using data from a single warehouse.

Secondly, data services are the way to liberate the data from specific applications and silos. State-of-the-art data integration solutions can turn your data into services, which can then be called upon by different applications, portals, and even mobile devices. Data services make your data more widely available, as you can reach it from anywhere, anytime.

Many data integration products are not really integration systems, but simply means of slow data warehouses. With the breakdown of these warehouses, you no longer have the data locked up in backend databases. Applications in the Cloud are interchanging data much more easily. This efficiency means that the closest future of data integration is now closely tied to the development of the data services approach.

December 17, 2010

Gartner Makes Its Predictions for Data Integration

Filed under: Data Integration — Katherine Vasilega @ 2:34 am

The year of 2010 is approaching its ending, and more and more predictions for 2011 are coming out to the public. It’s time to see what we have accomplished and what we are heading for in the area of data integration management. Good news is that Gartner published a report “Predicts 2011: Master Data Management Is Important in a Tough Economy, and More Important in Growth.” Let’s see what kind of MDM predictions for 2011 analyzed by Gartner experts it includes:

1. From 2009 through 2014, MDM software markets will grow at a Compound Annual Growth Rate (CAGR) of 18 percent, from $1.3 billion to $2.9 billion.

Data integration and MDM is a fast growing software market. The growth provides a major business opportunity for software vendors that specialize in these areas. The rapid growth of the market means that skilled MDM resources are in great demand among software and service providers. As a result, end-user organizations will struggle to adequately resource their MDM programs.

2. By 2015, 10 percent of packaged data management implementations will be delivered as software-as-a-service in the public Cloud.

Data integration software vendors will seek to leverage Cloud computing. Once organizations gain more experience with the public and private Clouds, the early adopters will seek to gain the same benefits with a wider range of software, including packaged MDM solutions.

3. Through 2015, 66% of organizations that initiate an MDM program will struggle to demonstrate the business value of it.

When IT departments initiate data integration projects, they often struggle to get the business on board and to demonstrate the business value of these projects. MDM needs to be guided by the business strategy, and will require strong involvement of business stakeholders and managers.

Today it is not enough to throw technology at the problem of inconsistent master data. Getting the proper governance, establishing the right data stewardship roles and responsibilities will be vital to the success of data integration initiatives. Meanwhile, I will watch for more predictions to appear and will try to suggest the most useful pieces of data for your consideration and review.

December 14, 2010

Data Integration Tools: A Point of Convergence

Filed under: Data Integration — Katherine Vasilega @ 7:56 am

Organizations are often adding business intelligence tools, rather than subtracting them. The problem is that with each tool you add, you also increase the complexity of your IT architecture, as well as the costs of your team’s training, software licensing and maintenance. This is especially true for data integration tools: ETL (extract, transform, and load), EAI (enterprise application integration), and EDR (enterprise data replication).

You can cut costs of your data integration initiatives if you review your data integration tools and their usage throughout your organization. ETL tools are built to move large amounts of data, while transforming it to match the business rules. EAI tools specialize in bite-sized, consumable pieces of information such as found in operational or transaction systems. EDR tools provide a mechanism to identify changes to datasets.

All three areas of usage are closely related to each other, which leads to the conclusion that the convergence of ETL, EAI, and EDR functionality is a good starting point for the modern data integration tool. Making a single tool available to diverse project teams for their data integration needs increases the productivity and cuts costs of data integration.

« Older Posts