Restricted access

April 29, 2011

Data Integration Challenges: Define Your Customer

Filed under: Data Integration — Katherine Vasilega @ 4:56 am

The IT and business alignment is a widely discussed challenge of data integration. The major data integration problem adds up to this: define customer.

Data from different functional areas doesn’t join up: sales orders are associated with the newly contracted customers, but the marketing campaign data is associated with prospects. Is a customer someone who’s actually bought something from you, or is a customer someone who’s interested in buying something from you? Should a definition include a certain demographic factor that reflects your typical buyer? If sales, marketing, service, and finance can all agree on a single definition of customer, then all the associated transactions could be easily integrated.

The thing is that all these specialists have their understanding of the word “customer”. That is why it is next to impossible for them to agree on a single definition and you have to somehow manage data integration without it.

To solve this issue, you can define what each functional area (and each CRM system) means by “customer”. This is how we know that customer data coming from a marketing system includes prospects, as well as existing customers. With this information, you can build a semantic model to understand how the different definitions of customer relate to one another.

Using this model, it would be possible to associate supply data with parts, cost data with product class, marketing data with brands, and so on. The relationships among these entities allow for data integration from different functional areas. This semantic model may be complex, but try to accept it and don’t head for simplifying it. The world is complex. Data integration requires a sophisticated understanding of your business, and standardizing vocabulary is not going to be the right answer to this challenge.

April 20, 2011

iPaaS: A New Trend In Data Integration?

Filed under: Data Integration — Katherine Vasilega @ 6:51 am

iPaaS (integration platform-as-a-service) is a development platform for building integration applications. It provides a set of capabilities for data integration and application integration in the Cloud and on-premises.

There are very few vendors offering iPaaS solutions at the moment. Although Gartner recognizes and uses the term, it still sounds confusing to researchers and data integration experts. So how does iPaaS work and can it benefit your data integration efforts?

Integration platform delivers a combination of data integration, governance, security and other capabilities to link applications, SOA services, and Cloud services. In addition to basic features that a Cloud solution should have, such as multi-tenancy, elasticity, and reliability, there are other capabilities relevant for iPaaS:

    1. Intermediation, the ability to integrate applications and services using the Cloud scenarios, which include SaaS and Cloud services, on-premises apps and resources.
    2. Orchestration between services, which requires connectivity and the ability to map data.
    3. Service containers to enable users publish their own services using either RESTful or SOAP technologies.
    4. Security covers the ability to authenticate and authorize access to any resource on the platform, as well as to manage this access.
    5. Enterprise Data Gateway installed on-premises and used as a proxy to access enterprise resources.

Data integration and application integration with and within the Cloud is the concept that business owners should consider nowadays. As of today, iPaaS would mostly appeal to companies that don’t mind building their own IT solutions or to ISVs that need to integrate Cloud silos they have created previously. It will be interesting to see whether iPaaS will become the next trend in the data integration discipline.

April 6, 2011

Salesforce Integration with QuickBooks: Out-of-the-box Solution on its Way

Filed under: Salesforce Integration — Katherine Vasilega @ 2:41 am and Intuit have signed a partnership agreement to provide Salesforce integration with QuickBooks to Intuit’s four million customers. The companies promise to finish developing the integrated solution in summer.

The solution is going to make CRM processes more convenient and transparent by displaying customer data along with financial information. Salesforce integration with QuickBooks will enable businesses to synchronize customer data in CRM with financial data in QuickBooks and QuickBooks Online. This will solve an issue of double data entry in two different systems.

Salesforce integration with QuickBooks will help small business owners to make better decisions. According to Intuit’s survey, more than 50% of small businesses perform CRM activities manually with pen and paper or with software, which is not designed for that.

With thousands of small businesses using both QuickBooks and, the integration of two systems is a great way to leverage the power of cloud computing and data integration strategies to help businesses grow.

April 2, 2011

Is Your Data Integration Technology Outdated?

Filed under: Data Integration — Katherine Vasilega @ 7:49 am

Spring is a good time to get rid of the old stuff and check out something new. This might as well be the time to upgrade your data integration tools. How can you learn if your data integration solution is outdated and should be replaced by something more productive? May be it just needs a little tuning? Here are the main check points to see if your solution’s performance still fits the industry standards.

Data transformation schemas
deal with both data structure and content. If data mappings are not as well-organized as possible, then a single transformation may take twice as long. Mapping problems can cause small delays that add up. The solution to the transformation issue is to make sure that data maps are written as efficiently as possible. You can compare your data integration solution to the similar ones to understand if the data transformation runs with the required speed.

Business rules processing are specific rules for the data that has to be validated. Too many rules can suspend your data integration processes. You have to make sure that the amount of rules in you data integration system is optimal, meaning that there are not too many of them running at the same time.

Network bandwidth and traffic—in many cases the performance is hindered not by the data integration tool itself, but by the size of the network you use. To avoid this issue, you need calculate the predicted performance under various loads and make sure you use the fastest network available for the data integration needs.

Data integration solution reminds a car: it can run but become slow if it is not properly tuned and taken care of. As we become more dependent upon the data integration technology, our ability to understand and optimize the performance issues will make a substantial difference.

March 10, 2011

The Key Data Integration Strategies for Successful CRM

Filed under: Data Integration — Katherine Vasilega @ 6:39 am

One of the great values data integration provides is a possibility to gain a deeper insight into one’s customers. It is not surprising that data integration with CRM (customer relations management) systems is one of the main directions in the industry development. As more companies choose managing customers electronically, it is quite useful to apply the most effective data integration strategies to pay-off for CRM investments.

The recent survey by the data integration experts and authors—Christopher Barko, Ashfaaq Moosa, and Hamid Nemati, —explores the significant role of data integration in electronic customer relationship management (e-CRM) analytics. They reviewed 115 organizations including both B2B and B2C companies and sorted out four data integration initiatives that provide for better CRM:

    1. Integrating more data sources. The research shows that the total value of CRM project increases when you integrate more data sources. As sales people are using more channels than ever before to reach prospects and customers, no wonder that data integrated from all these channels is more efficient, than when stored in the isolated silos.

    2. Integrating offline data with online data gives a better picture of customer’s buying habits. 62 percent of respondents said they integrated these data sources, while 30 percent did not. Not surprisingly, those who integrated the online and offline data experienced greater value from their e-CRM projects.

    3. Integrating external data (e.g., from social media sites) into the central repository. 74 percent integrated external data in some form, while 26 percent did not. The companies that practice external data integration in their e-CRM projects enjoy significantly more benefits.

    4. Using a centralized data warehouse or a CRM-specific data repository does provide a deeper customer insight. Those who used a decentralized data repository (legacy databases, operational data stores) experienced significantly less benefits than those who centralized their data storage.

As the number of marketing channels used to communicate with customers continues to multiply, so does the number of places used to store the data. The research reveals that the most efficient data integration strategies include integrating different kinds of data from multiple channels and keeping it in the central repository. These data integration best practices help ensure marketing efforts have a positive effect on sales.

March 5, 2011

How Can Data Governance Serve Data Integration Projects?

Filed under: Data Integration, Data Quality — Katherine Vasilega @ 3:56 am

Data governance initiatives in an organization are intended to cover data quality, data management, and data policy issues. These activities are carried out by data stewards and a team that develops and implements business rules for administrating the use of data.

The focus on data governance is essential when the company has to implement a successful data integration strategy and use it for analysis, reporting, and decision-making. Here are some ways of making data integration projects more efficient with data governance:

    • It brings IT and business teams together. Data governance identifies what is really important to the business and helps establish business rules that are crucial for data integration.

    • A data governance program can help your company define and measure the potential ROI you get from maintaining data. You can use this information to calculate the ROI for data integration projects.

    • It helps you learn who’s responsible for the data quality. Data governance provides valuable information that enables to appoint data stewards and decision makers for data integration projects. Since data governance tells you who’s responsible for the data, you know where to go to resolve data quality issues.

    • Data governance can save you money, because it helps establish best practices and select cost-effective data integration and data quality tools.

Data governance and data integration are tightly connected with each other. You are not likely to enjoy data integration benefits without a strong governance program. On the other hand, data governance is only possible if your data is stored in an integrated system. My advice: make sensible use of both.

February 24, 2011

What Is The Difference Between Data Conversion and Data Migration?

Filed under: Data Migration — Katherine Vasilega @ 8:28 am

The terms data conversion and data migration are still sometimes used interchangeably on the internet. However, they do mean different things. Data conversion is the transformation of data from one format to another. It implies extracting data from the source, transforming it and loading the data to the target system based on a set of requirements.

Data migration is the process of transferring data between silos, formats, or systems. Therefore, data conversion is only the first step in this complicated process. Except for data conversion, data migration includes data profiling, data cleansing, data validation, and the ongoing data quality assurance process in the target system.

Both terms are used as synonymous by many internet resources. I think the reason for that might be that there are very few situations when a company has to convert the data without migrating it.

Data conversion possible issues

There are some data conversion issues to consider, when data is transferred between different systems. Operating systems have certain alignment requirements which will cause program exceptions if these requirements are not taken into consideration. Converting files to another format can be tricky as how you convert it depends on how the file was created. These are only few examples of possible conversion issues.

There are some ways to avoid data conversion problems:

    1. Always transform objects into printable character data types, including numeric data.
    2. Devise an operating system-neutral format for an object transformed into a binary data type.
    3. Include sufficient header information in the transformed data type so that the remainder of the encoded object can be correctly interpreted independent of the operating system.

Data conversion is often the most important part of data migration. You have to be very careful during this stage to assure data quality in your target system.

February 17, 2011

Data Integration in SharePoint 2010

Filed under: Data Integration — Katherine Vasilega @ 6:23 am

A survey by AIIM (Association for Information and Image Management) states that although SharePoint is being rapidly adopted by organizations, at least half of the companies that are implementing the platform don’t have business use in mind.

This might be a reason we don’t see millions of companies shifting their data integration initiatives into SharePoint. It may be only a question of time, as SharePoint 2010 comes with rich integration capabilities. Here are some of the features that can be leveraged for external data integration and application integration:

    1. Business Connectivity Services (BSC) is a new feature of the SharePoint platform that provides new means for external data integration into SharePoint 2010. It enables to create connections to external data sources through the use of SharePoint Designer or more complex scenarios with custom code development.

    2. Web Services can be leveraged by both SharePoint and external systems for data integration and application integration purposes. Common services include the ability to authenticate, search, and manage content. SharePoint 2010 also includes built-in RESTful Web services, which allows the integration of remote systems.

    3. Client Object Models are used to integrate SharePoint and other systems to provide a better usability. SharePoint 2010 introduces three new client API’s: ECMAScript Client, SilverLight Client, and .NET Managed Client. These object models enable users to access both SharePoint and other data sources from a single interface that does not have to be or look like the SharePoint interface.

    4. The CMIS (Content Management Interoperability Services) connector for SharePoint 2010 enables to perform content management functions between systems that comply with the CMIS specification.

There are many ways in which organizations can leverage SharePoint for their data integration needs. Nevertheless, the question on whether companies will start data migration and data integration into SharePoint 2010 in the nearest future remains open.

January 3, 2011

The Additional Value of Data Integration

Filed under: Data Integration — Katherine Vasilega @ 2:50 am

The data integration discipline has undergone an impressive expansion over the last decade. Up-to-date data integration practices influence various business processes across organizations. I have talked a lot about the value of data integration for decision-making, sales, and customer service. What else can data integration add to the way the business operates? Here are some thoughts on the key issue.

Green Technology

Recent climate changes and the rising cost of electricity have led many people to reconsider the environmental sustainability of data centers. Data centers today are reducing power consumption and server rooms by consolidating redundant data and virtualizing hardware servers. Data integration tools and techniques help these consolidations and make IT more ecologically friendly.

The data integration technologies can reduce the number of redundant databases, thereby cutting down on the number of servers, plus the budgets and resources they consume. Furthermore, data integration techniques, such as data federation and data services, can assemble data sets on the fly, without generating permanent databases that burn up server resources.

Collaboration and Education

Collaboration between the IT and business teams is a must for data integration initiatives. The collaboration among the technical and business people involved in data integration projects helps them learn more about the specific tasks of each party involved in these projects, as well as understand the value business and IT teams bring to the organization.

The best practices listed above help organizations achieve more far-reaching uses of data integration tools and techniques. Getting beyond the standard definition of data integration value brings a wider perspective to the business development.

December 30, 2010

Data Services in Data Integration

Filed under: Data Integration — Katherine Vasilega @ 3:51 am

Many data management professionals agree that the increasing volume and complexity of data require a new approach to data integration with its traditional tools and methods. Organizations today are expanding their usage of information-as-a-service (also called data services) to support new business requirements, such as real-time data integration, creating data domains, improving security, integration with unstructured data and external sources.

Let’s take closer look at a data services approach in the data integration. First of all, it helps bring in data from more sources. This provides a better presentation of the important information for decision-making, bringing a return on investment in a shorter period time, compared to using data from a single warehouse.

Secondly, data services are the way to liberate the data from specific applications and silos. State-of-the-art data integration solutions can turn your data into services, which can then be called upon by different applications, portals, and even mobile devices. Data services make your data more widely available, as you can reach it from anywhere, anytime.

Many data integration products are not really integration systems, but simply means of slow data warehouses. With the breakdown of these warehouses, you no longer have the data locked up in backend databases. Applications in the Cloud are interchanging data much more easily. This efficiency means that the closest future of data integration is now closely tied to the development of the data services approach.

« Older Posts