Agile Data Governance: Solving Enterprise Data Quality Problems


Written by Paul Kirvan on October 28, 2008 in Business Continuity.

Companies today are under increasing pressure to make better business decisions in less time, with less risk, while producing higher quality results. Their challenges are enormous, as are the many issues that can arise and potentially jeopardize their success. Among the most pervasive problems companies face is the consistently poor quality of the internal data that they are using to draw conclusions and make decisions.
Poor data quality is not a new problem, but now solving it is easier than before because companies no longer have to rely on methods that require them to “boil the ocean.” Streamlined approaches to data governance that incorporate new processes and data stewardship technologies enable more agile methods for improving data quality.

Problems With Top-Down Data Governance

The problem with traditional data governance programs has been that most companies were taking a top-down approach, while more pressing short-term business demands were derailing efforts and distracting resources.  Executives responsible for these programs and line managers, whose cooperation were needed to make the processes successful, were always scrambling to make their own revenue numbers, launch new products and meet other required business objectives, rather than focusing on the often laborious data governance process, which only produced intangible results.

A Better Approach: Agile Data Governance

An alternative to top-down data governance, and a much better way to address the problem, is for companies to adopt a more agile approach.  By introducing more agile processes, companies can achieve quick wins by implementing data governance processes and policies in smaller pieces, learning from and adapting the approach with each segment, taking time off in between increments to focus on pressing business goals, and then coming back together to address another data domain, while making steady progress over time.  For example, with an agile approach, a company could decide to focus its efforts on a master data management project for its customer data that it could actually solve within 6-8 months, which is quite different from a top-down approach that takes years just to get started!

Companies that want to try an agile data governance approach should follow a number of guidelines that will assist them in a successful implementation:

  1. Determine up front which data fixes are going to deliver the greatest business benefits and focus on fixing those first; this will help guarantee that the project is a success.
  2. Limit the size of the data governance team, and let the team evolve as different data is being addressed at each stage of the project; this will help streamline the process and eliminate many of the potential political issues.
  3. Assemble a solid team of data stewards for each piece of the project to help ensure success.
  4. Select a small, core data governance board made up of executives who can authoritatively represent the business goals of the entire organization. The board needs to have enough visibility to be able to sort out the biggest data problems the enterprise faces and determine which “critical few” problems should be tackled first, including the very first project, which should deliver the biggest overall bang for the buck.

Steps to a Successful Data Governance Project

  1. Select the data governance implementation team for the first project – once data priorities have been set, the core executive board should select the team that will manage the first piece of the project.
  2. Define the data problem – the data governance implementation team should evaluate and determine the size and scope of the data problem they are going to address.
  3. Draft the data steward team – the data governance team should create a team of data stewards made up of those individuals with the most knowledge of the data being addressed in each piece of the project.  The data governance board should provide “air cover” to free data stewards up for this task, and set aggressive deadlines to avoid analysis paralysis.
  4. Validate or disprove assumptions of the data governance team – the first job of the data steward team will be to investigate data quality problems in the first chosen data domain.  The job of the data stewards is to determine the extent of data quality issues, root causes of data problems, and the extent of potential damage when conducting a proof of concept on the data.
  5. Establish policies for making changes to data – when the data steward team delivers the results of their analyses, the data governance team needs to step back in to create new policies for data and to secure board approval for their recommendations.
  6. Enlist internal IT groups required to fix problems – the data governance and steward teams will work together to kick off the actual project implementation and select enterprise architects from internal IT groups to help design the best solution.
  7. Compare results, evaluate and determine next domain – After testing the solution, bringing data into compliance across all participating systems, and implementing new data quality tools, the team should compare the results and exceptions with the new business policies to determine where issues still exist (it may take several iterations to cure a single data domain completely).

Tags: ,