Restricted access

October 6, 2009

Data Integration Steps out of Extract, Transform, and Load (ETL) Borders

Filed under: Data Integration, Data Migration, ETL, Open Source — Tags: — Olga Belokurskaya @ 8:12 am

We all got used to speaking of data integration in terms of extract, transform, and load (ETL) mostly. And that’s pretty fair, for gathering and transforming data from one location and putting it into another location has always been, and still is the major task for data integration.

However, according to Rick Sherman, a data management expert, new trends make data integration step out of mere ETL borders, as technologies and processes evolve helping data integration tools turn data into “comprehensive, consistent, clean and current information.” Many tools support processes like data migration, data profiling and quality, application consolidation, etc. The time when IT departments had to build those processes into data integration, have passed, and tools appeared with all above mentioned functions pre-built.

Thus, enterprise data integration initiatives that once were extremely time-consuming more and more tend to become real-time as business demands more current information.

One more thing, Sherman dwells on is hand-coding being an out-of-date practice. Frankly, why keep using error-prone hand-coding, when there is a wide range of ETL tools available. There is a choice in configuration and price making it possible to find the tool to fit one’s needs. Taking into account open source ETL tools available almost for nothing, the devotion to hand-coding seems pretty strange or, at least unwise.

October 1, 2009

Open Source Data Integration: Numbers and Predictions

Filed under: ETL, Open Source — Olga Belokurskaya @ 8:34 am

Recent survey by Gartner showed increased interest to open source data integration. About 11 percent of organizations dealing with data integration technologies have evaluated open source tools along with commercial products.

However, this doesn’t mean that open source data integration tools are equal with commercial tools in terms of deployments. And it’s early to make any predictions about when the situation will change or will it change at all.

Most of organizations using open source data integration tools use them for ETL purpose to support business intelligence and data warehousing projects. The reason is that they are less mature capabilities for supporting other styles of data integration, including data federation and real-time data integration techniques, according to Ted Friedman, an analyst at Gartner.

However, there is a tendency for open source data integration tools to move from mere ETL or data integration to data quality. Some analysts are optimistic enough to see here a movement to master data management (MDM), something open source data integration tools are not yet supposed to be used for.

September 14, 2009

How to Reduce Total Cost of Ownership of Data Integration Software

Filed under: Data Integration, Open Source — Tags: , — Olga Belokurskaya @ 8:07 am

When choosing a data integration solution, it is important to keep in mind a lot of things, the most crucial being that while many companies tend to underestimate the TCO (comprising license fee, hardware costs, and labor costs) of data integration, for every dollar spent on integration software, enterprises spend $6 on subsequent implementation and support. This goes for proprietary software.

Under tough economic conditions for many companies the total cost of ownership of enterprise of proprietary data integration solutions is becoming prohibitive. One may start to think of data integration as of something that constantly consumes enormous resources, both human and financial, and that would make a point. But another point is that to reduce the total cost of ownership, you have to change the cost structure, that’s it:

  • Today that there is a number of open source solutions available for evaluation and real-world projects at no charge. With open source, you can practically eliminate licensing fees at all.
  • As far as hardware costs are concerned, eliminating the vendors whose solutions do not provide the acceptable level of openness can result in a considerable saving.
  • When it comes to labor cost, there are two main criteria.
    • The first of them is whether the solution is straightforward enough for a non-trained user to use, and whether it is effective, which means that the user is able to do the job in minimal time.
    • The second criterion is the flexibility of the product, the extent of reusability of its configuration for follow-on tasks.

    Commonly, open source manages to provide both, oriented on and supported by a large community that aims to consistently enhance the development.

Learn more about how to reduce the total cost of ownership from our “Guide to Reducing Data Integration Costs.”

September 7, 2009

Why Enterprises Go Open Source for Data Integration?

Filed under: Data Integration, Open Source — Tags: , — Olga Belokurskaya @ 12:37 am

open source data integrationSome ten years ago, using open source was unlikely for big serious companies. Now that the benefits of freely distributed software have become more evident than ever and an abundance of such products has appeared at the market, more and more companies go open source.

Open source data integration is no exception. Freely available solutions are doubly beneficial, bringing the license cost to minimum and enabling companies to save dramatic amounts on maintenance. Organizations rely on open source when it comes to integrating their data. But it is not only about money. Open source also has a number of advantages over traditional software such as:

- Better performance and reliability
Open source solutions have vast communities of developers, which ensures testing all the functional range of a product on different platforms before releasing, bugs are found and fixed rapidly, enhancements to the code are also easier to make due to the availability of the source code.

- Multi-platform support
Typically, open source software supports numerous platforms, leaving it to the user to choose the one that fits their requirements better – something many proprietary software solutions cannot offer.

- Higher level of security
With the source code publically available, open source software typically suffers fewer vulnerability attacks than proprietary solutions.

- Flexibility
Most open source developments allow a tremendous scale of flexibility and can be reused in a vast range of cases with little to none customizing required.

- Easier deployment
There is a tendency for open source software to concentrate on the essential features instead of implementing dozens of secondary features that hardly anyone uses. Due to that such software is usually more straightforward in use than proprietary products.

You can learn more about why enterprises go open source for data integration from the “Guide to Reducing Data Integration Costs.”

August 28, 2009

Open Source vs Saas. Define the difference

Filed under: Open Source — Olga Belokurskaya @ 6:17 am

Open source and SaaS solutions both can reduce initial costs and use subscription-based pricing model, and traditionally are considered to be alternatives to costly proprietary applications. Taking into account the fact that SaaS applications are very often open source at heart, then, what is the difference between them? An article at defines it.

According to Gartner and Forrester experts, the difference is seen when we speak about hidden costs, customization, and resources.

What’s hidden behind?

  • The bitter truth is that initial costs are rather small compared to all the maintenance, correcting failures, addressing the occurring issues, and other support work. As Forrester research showed, the cost of open source solutions support may be high, because it is quite resource-intensive.
  • The challenge with SaaS is, that though the support costs are minimal, in case of any downturn of failure, or any other trouble with the environment, company’s resources can’t fix it and have to wait until somebody from SaaS provider’ comes and helps. This is because a company has no full control of its SaaS application.

Demand for resources

  • An expert from Gartner advices to address to SaaS in the situation when IT resources are pretty tight. SaaS providers offer, normally, packaged support so company’s IT department is free from the burden of SaaS applications upgrades.
  • Open source solution demand for some additional work to be done to receive support from the open source communities. But this support includes many options:  updates, bug fixes, upgrades and enhancements

What about customization?

  • Well, speaking about customization, SaaS applications are not the better choice. Deep customization is not possible for SaaS solutions. So, if an organization looks for a solution to suite its special needs, SaaS is not likely to help.
  • The ability to fully customize makes open source applications appealing. Open source permits totally customize applications to fit any organization’s needs. Moreover, many open source providers offer on-demand and on-premise options. This gives a company more flexibility, for they may begin with using on-demand option, and as the use of the solution becomes more mature, it may be brought in-house. And there is no such flexibility for SaaS.

August 18, 2009

Reducing Data Integration Costs with Open Source ETL

Filed under: Data Integration, Open Source — Tags: , — Olga Belokurskaya @ 1:53 am

Right data integration and data quality are critical points for companies wishing to have fast time-to market and to manage complex sales and marketing programs. However, when budgets are limited it becomes difficult to cope with increasing expenses of data integration. In fact, the amount that an enterprise spends on Extracting, Transforming, and Loading (ETL) may reach the numbers of millions of dollars, what makes executives (especially non-ITs) clutch their heads in horror and spend sleepless nights analyzing whether their ETL data-integration technology is as beneficial as it is positioned.

Such factors as license costs, labor costs, and hardware costs drive data integration cost up. As the projects become more complex and amount of data increases taking a good care about data becomes more and more expensive.

Here are some best practices on how to lower data integration costs:

  • Consider using commercially supported open source tools for your integration projects.
  • Verify licenses to make sure the product you’re using is really an open source solution and the terms of license suit you.
  • Openness of the source code is an advantage of open source over proprietary software. You are always free to view, fix, and modify the code to make your open source tool better suit your particular requirements.

However, think over all the pros and cons. License costs are not the only expense when implementing software. If an open source solution is difficult to learn to use or implement, you better think whether it’s worth it.

Look through our Guide to Reducing Data Integration Costs to discover more.

August 5, 2009

Open Source BI Gains Popularity

Filed under: Open Source — Olga Belokurskaya @ 1:04 am

As IT budgets are still tightened, companies turn more willingly to open source BI software, claims an article at

Of course, that doesn’t mean companies are throwing away their proprietary solutions and replacing them with purely open source ones. Open source software is a great means to complement commercial BI deployments and filling occurring functionality gaps. And in case when new business requirements emerge asking for appropriate new BI solutions in the conditions of exhausted budgets, open source software can be a savior.

However, there are still some fears about open source BI deployment:

  • The lack of internal expertise for customizing OS software to interact with commercial BI software as well as underlying data sources and applications

  • Open source BI vendors’ viability.

Well, the firs fear has sense, for without the internal expertise, open source BI deployments can quickly get out of hand. As for the second fear, however, is doubtful. Open source software, thanks to its free code, is available to large communities of developers, which means that even if the vendor itself goes out of business, the product is more then likely would still be maintained and even improved. The same can’t be said about commercial BI vendors and products.

June 25, 2009

Open Source Data Warehouses: the Benefits

Filed under: Data Warehousing, Database Integration, Open Source — Tags: , — Olga Belokurskaya @ 1:21 am

Open source data warehouses possess the same options as any other types of open source software: the same model of licensing, community development processes, and same degree of openness. They may be offered as free downloads, or for a nominal flat fee, as fully supported systems. Or there may be no limit to the number of licenses and implementations a company may make with the software.

Acording to BeyeNetwork article, the benefits of the open source data warehouses are following:

  • Up front and maintenance costs are less than those of proprietary software. Besides, there is a possibility to customize the products companies use to improve their operations, for the original source code is open and may be downloaded.
  • Skill sets that are widely available in the market are employed.  As a result, an organization with existing database or data warehouse expertise will not have to look further when a new open source data warehouse project is put into place.
  • Improved standardization. Transparent and community supported open source code considers important standards to be consistently supported across all versions and implementations. Something that proprietary formats cannot and will not offer.
  • Flexibility which enables enterprises to expand the solutions to an unlimited number of users, with no per-user or per-processor charges of proprietary software packages.
  • Community effect. Open source solutions leverage communities of developers and innovators to advance development. New code and features are contributed back to the community, constantly increasing the range of new options available to end users.  Moreover, companies may address the community in order to fix any bugs or security flaws, which takes, normally, only days, instead of waiting weeks and months for the next security patch or service pack from a vendor.
  • Incremental implementation.  There is no need to a mega project at once. Projects can start small and build upon the success of implementations. This dumps the tendency to “overpromise,” which is often a necessary evil for acquiring optimal levels of funding for data warehouse projects.

June 24, 2009

Getting Most out of Open Source Data Warehouse

Filed under: Data Warehousing, Open Source — Tags: — Olga Belokurskaya @ 5:53 am

There’s been quite a period of time since open source data warehouses evolved and gained popularity. However, an open source data warehouse is still regarded as a solution for small or mid-sized companies lacking enough budgets for solid proprietary solutions. Bigger companies may also use open source solutions as complimentary to their proprietary data warehouses.
Getting the most out of an open source data warehouse implementation is possible. There are some ways below provided by Claudia Imhoff:

  • Open source data warehouses complementing already existing proprietary enterprise solutions may help quickly address the new company’s needs. Proprietary solutions being more strategic are not so fast to react to those changes.
  • Normally, it’s the analysts who work with data warehouses; they are familiar with building massive queries and other technical stuff. But in some cases, there are end users who don’t have special technical knowledge, and need as much ease of use as possible.
  • Open source data warehouses should be compatible with related open source environments.
  • While open source data warehouses may seem cheaper than proprietary solutions at first, additional costs, such as transition and training costs, should be taken into account.

May 15, 2009

Apatar at 2009 Community Choice Awards

Filed under: Open Source — Olga Belokurskaya @ 8:19 am

Last week SourceForge started accepting nominates for their fourth annual Community Choice Awards to recognize projects and their authors who have excelled in the following software categories:

Best Project
Best Project for the Enterprise
Best Project for Gamers
Best Tool or Utility for SysAdmins
Best Visual Design
Best Tool or Utility for Developers
Best Commercial Open Source Project
Best Project for Academia
Best Project for Multimedia
Best Project for Government
Most Likely to Change the Way You Do Everything
Best New Project

Users may nominate the projects they think are the best until May 29th, and then ten projects with the most nominations in each category will become finalists. The winners will be announced on on or around July 23, 2009.

 Apatar will also take part in the event. So, if you like the project, click the button below and nominate us!

« Older PostsNewer Posts »