Restricted access

May 6, 2009

Migrate Your Data in Seven Steps

Filed under: Data Migration, Data Quality — Tags: , — Olga Belokurskaya @ 1:52 am

Historically, data migration projects have had a tendency to fail; according to Bloor Research, as many as 60 percent do not succeed.  How to prevent failure when migrating your data? Here is a seven-step process:

  1. Source system exploration. The first phase of a data migration project is to identify and explore the source data. The most appropriate route for identification is to group data, customer names, addresses and product descriptions based on the target model.
  2. Data assessment. The next logical phase is assessment of the quality of this source data. If the new system fails due to data inconsistencies, incorrect or duplicate data, there is very limited value in migrating data to the target system. To assess the data, profiling the data is highly recommended.Through the use of data profiling, you can:
    •  Immediately identify whether the data will fit the business purpose.
    •  Accurately plan the integration strategy by identifying data anomalies up front.
    •  Successfully integrate the source data using an automated data quality process.
  3. Migration design. In the migration design phase, the main tasks will be to define the technical architecture and design of the migration processes.  In addition, you will define the testing processes and how to transition them to the production system. You will also determine whether there will be a parallel run, a zero-downtime migration, or whether you will be expecting to complete the migration and simply “decommission” the old system.
  4. Migration build. Be careful about using a “just enough” development approach with your data migration, simply because the migration will be executed only once and there will be limited reuse of the code. These assumptions explain why data migrations are prone to failure.
  5. Execution. After comprehensive testing, the time comes to run the migration. In the majority of cases, the source systems are shut down while the migration executes. To minimize impact, this is likely to occur over a weekend or public holiday. In some cases, where the source applications are required to run 24/7, 365 days a year, a zero-downtime migration approach may be needed.
  6. Transition. At some point after the data has been migrated, decide when to move to the new system, and where appropriate, retire the old system. During the execution phase, audit trails and logs will be created to ensure that all data has been correctly migrated and, when appropriate, that the correct synchronization has been achieved. Finally, after reviewing the audit trails and logs, you will be prepared to make the decision to transition users to the new system.
  7. Production. As part of the design process, a system retirement policy will be created to address the old system. There will also be ongoing data quality enhancements. Remember that because not all source systems will be retired, data quality issues may be identified and will need rectifying in several of the source systems. You will also need to manage ongoing improvements and monitor the data quality of the new system.

No Comments »

No comments yet.

RSS feed for comments on this post. TrackBack URL

Leave a comment