Restricted access

December 18, 2009

On a Couple of Misconceptions About ETL Tools

Filed under: Data Integration, ETL, Open Source — Tags: , — Olga Belokurskaya @ 8:20 am

When deciding to start a data integration process, many companies consider using ETL tools instead of hand-coding. Such a decision is justified by the fact – and many data integration experts agree with it – that hand coding is error prone, takes time and additional resources, etc. However, it’s also wrong to assume that an ETL tool will help to finish data integration project sooner, or will result in some substantial cost savings, according to a TDWI.

Their point is that though ETL tools definitely accelerate the process of data integration at some level, one should not leave aside time that is to be spent on ETL tools evaluation, selection, and implementation.

Another deception is about cost savings. The acquisition cost of ETL tools is quite sufficient, and the annual support cost is often overlooked when a decision is being made on selection and implementation of an ETL tool. Thus, companies have a bit wrong idea about the amount of savings they might have.

The misconceptions described above, are a source of inappropriate expectations, and as a result, wrong assessment of data integration initiative expenses, and at worst, failed data integration initiative.

I think, the situation’s a bit different, when we speak about open source ETL tools. First, there’s no such thing as annual support cost. Huge developer and user communities make it possible to receive support from other users, without paying for it. Then, license costs of open source ETL solutions are really low, which allows to redirect the released budget where there would be a demand for additional finance. So, here I see a real possibility to reduce the cost of data integration with the help of ETL tools.

What I agree with, is that selection process will take time, as well as deployment (including user training), though open source solutions are typically easier to deploy, compared to proprietary ETL tools. Companies should take time for proper evaluation of ETL tools, either open source or proprietary; and I do agree that the decision should be taken based on whether an ETL tool fits this peculiar company’s business needs best and is capable to provide a company with help in achieving their goals.

December 9, 2009

Open Source ETL and BI Tools Deployments to Grow Says Gartner

Filed under: Data Integration, ETL, Open Source — Tags: , — Olga Belokurskaya @ 10:25 am

In one of their latest press releases, Gartner made some interesting predictions regarding open source ETL and other kinds of BI tools. Though it’s early to put open source data integration platforms on the same level with complex proprietary solutions, the deployment of open source tools shows solid growth.

Gartner has pointed out the fact that open source ETL and BI tools are more frequently used by mid-sized businesses, governmental and public sectors. And many ISVs use open source BI solutions as the additional functionality to their own applications.

What is even more interesting: large vendors, proprietary commercial tools providers seriously care about finding the ways to address the challenges from open source data integration and BI offerings, as the latter become their lower-cost competitors.

Assuming this, I think that open source data integration, ETL and other kinds of business intelligence solutions have earned confidence from the business side, and are expected to become even more widely used in the nearest future.

December 7, 2009

Choosing ETL That Fits Your Business Requirements II: Consider Open Source

Filed under: Data Integration, ETL, Open Source — Tags: , — Olga Belokurskaya @ 1:49 am

I suppose this posting to be a kind of a continuation of the previous one. Here I’m again about ETL and data integration solutions selection, but I’d like to concentrate on open source ETL. I won’t make any discovery if, again, repeat that today’s open source solutions are good enough for ETL operations, and  data integration and BI experts are expecting them to develop into solutions for master data management.

But today, open source ETL provides alternative to proprietary solutions which are usually costly and supposed to be used for more complex data integration processes, apart from mere ETL. However, for mid-sized and small businesses that, as a rule, have smaller budgets and smaller open source ETL solutions are a means to address their data integration needs.

But I was about business requirements, or rather how open source ETL tools may address company’s business requirements for data. Here I see several ways:

First, if a company by chance has a couple of their own developers, they could make necessary customization to company’s ETL, thanks to the availability of the code.

Then, as a rule, open source solutions are supported by developers’ communities, some of which are really powerful. So, the community behind the open source ETL that a company uses may help with needed functionality or customization of existing ones to meet company’s business requirements.

And don’t forget about the vendor itself. A company may address directly to the vendor of their open source ETL and require additional functionalities that meet company’s peculiar needs.

And, as a rule, any of the actions described above will cost less and the result will take less time to deliver than in case with proprietary data integration tools.

Well, though the posting sounds so bright, there still may be issues with open source, such as vendors that stop supporting their solutions, etc. However with the communities behind, and thanks to the openness of the code, the chances to overcome those issues seem to me higher than in case with proprietary solutions.

November 16, 2009

Open Source Data Integration and Quality Tools: Being Reasonable

Filed under: Data Integration, Data Quality, ETL, Open Source — Tags: — Olga Belokurskaya @ 2:40 am
While open source technologies gain more popularity as relatively cheap options compared to commercial data management tools, it should be clear that some of the open source tools are more mature, than others, an thus, take care and special attention to what open source may offer for data management, what your needs are, and whether the offerings answer all your needs.

Thus, as most of open source BI software that is now on the market may adequately address the needs of not just SMBs, but quite large companies, many analysts regard them as mature tools.

Almost the same may be said about open source data integration, as there are several mature enough offerings for ETL operations. Moreover, the steps are done toward other forms of data integration.

But, as for data quality tools, open source offerings are still too weak to go as independent option, and not as an addition to proprietary software. Today, such tools may serve as relatively inexpensive means of data profiling.

So, all the above was about to remind of being reasonable and analyzing your needs. While open source may offer mature solutions, not all open source tools are equal, as claims

September 14, 2009

How to Reduce Total Cost of Ownership of Data Integration Software

Filed under: Data Integration, Open Source — Tags: , — Olga Belokurskaya @ 8:07 am

When choosing a data integration solution, it is important to keep in mind a lot of things, the most crucial being that while many companies tend to underestimate the TCO (comprising license fee, hardware costs, and labor costs) of data integration, for every dollar spent on integration software, enterprises spend $6 on subsequent implementation and support. This goes for proprietary software.

Under tough economic conditions for many companies the total cost of ownership of enterprise of proprietary data integration solutions is becoming prohibitive. One may start to think of data integration as of something that constantly consumes enormous resources, both human and financial, and that would make a point. But another point is that to reduce the total cost of ownership, you have to change the cost structure, that’s it:

  • Today that there is a number of open source solutions available for evaluation and real-world projects at no charge. With open source, you can practically eliminate licensing fees at all.
  • As far as hardware costs are concerned, eliminating the vendors whose solutions do not provide the acceptable level of openness can result in a considerable saving.
  • When it comes to labor cost, there are two main criteria.
    • The first of them is whether the solution is straightforward enough for a non-trained user to use, and whether it is effective, which means that the user is able to do the job in minimal time.
    • The second criterion is the flexibility of the product, the extent of reusability of its configuration for follow-on tasks.

    Commonly, open source manages to provide both, oriented on and supported by a large community that aims to consistently enhance the development.

Learn more about how to reduce the total cost of ownership from our “Guide to Reducing Data Integration Costs.”