Common Enterprise wide Data Governance Issues – #12. No Enterprise wide Data Dictionary.

This post is one of a series dealing with common Enterprise Wide Data Governance Issues.    Assess the status of this issue in your Enterprise by clicking here:  Data Governance Issue Assessment Process

No Idea What This Means

Anyone know what this acronym means?

An excellent series of blog posts from Phil Wright (Balanced approach to scoring data quality) prompted me to restart this series.  Phil tells us that in his organisation, “a large amount of time and effort has been applied to ensure that the business community has a definitive business glossary, containing all the terminology and business rules that they use within their reporting and business processes. This has been published, and highly praised, throughout the organisation.” I wish other organisations were like Phil’s.

Not only do some organisations lack “a definitive business glossary” as Phil describes above, complete with business rules….
Some organisations have no Enterprise wide Data Dictionary.  What is worse – there is no appreciation within senior management of the need for an Enterprise wide Data Dictionary (and therefore no budget to develop one).

Impact(s):

  • No business definition, or contradictory business definitions of the intended content of critical fields.
  • There is an over dependence on a small number of staff with detailed knowledge of some databases.
  • Incorrect or non-ideal sources of required data are identified – because the source of required data is determined by personnel with expertise in specific systems only.
  • New projects, dependent on existing data, are left ‘flying blind’.  The impact is similar to landing in a foreign city, with no map and not speaking the language.
  • Repeated re-invention of the wheel, duplication of work, with associated costs.

Solution:

CIO to define and implement the following Policy:  (in addition to the policies listed for Data Governance Issue #10):

  • An Enterprise wide Data Dictionary will be developed covering critical Enterprise wide data, in accordance with industry best practice.

Does your organisation have an “Enterprise wide Data Dictionary” – if so, how did you achieve it?  If not, how do new projects that depend on existing data begin the process of locating that data?  Please share your experience.

My interview with Dylan Jones

Dylan Jones of DataQualityPro interviews me about the process I use to assess common Enterprise wide data issues. Use this process to assess the status of data governance within your organisation or that of a client.

Data Quality Pro interview with Ken O'Connor Data Consultant

Russian Gas Pipe and Data Governance

As you know, Russia supplies Gas to many European countries.

What's flowing through your critical data pipelines?

Do you know what’s in your critical data pipelines?

Could you imagine Italy purchasing gas from Russia without checking what exactly was flowing through the pipe?  I’m no expert on gas pipelines, but I know that before completing the agreement to purchase the gas, Italy and Russia would have agreed metrics such as:

  • Volume of Gas
  • Calorific value (Energy content)
  • etc.

So what? What else would one expect?  Applied common sense… yes?

Why is it that such common sense is often lacking in Data Migration and Data Population projects?  Why do some Enterprises continue to perform data population of, and ongoing data entry to, critical data repositories without fully understanding the data they are pumping into the repository?

A simple example involves Date of Birth.  The business ask the IT function to populate Date of Birth in the new AML / BASEL II / CRM / other repository. Some time later, when data population is complete, the business begin to express concerns:

  • “We never realised we had so many customers aged over 100 ???”
  • “I thought we had more Student customers”
  • “How come so many of our customers share the same birthday ?”
  • “These are not the results we expected”
  • etc.

Performing data population on the basis of what the source data “should contain”, without analysing what exactly it does contain is known as ‘Load and Explode’ approach to Data Population.  I cover this Enterprise Wide Data Issue in more detail here.

We in the “Data Governance”, “Data Quality” industry need to educate the business community on the “common sense” parts of data governance, and the need to engage “Data Governance Professionals”  to ensure that “Data Quality Common Sense” is actually applied.

Feedback welcome – Ken

Common Enterprise wide Data Governance Issues #10: No ‘Master’ repository of business rules

This post is one of a series dealing with common Enterprise Wide Data Governance Issues.  Assess the status of this issue in your Enterprise by clicking here: Data Governance Issue Assessment Process

Business rules provide critical details about data fields, including the ‘business’ name of the field, the business purpose of the field, the values it may hold, the business meaning of each value, and interdependencies with other data.

An example of a business rule could be ‘Account Type must be consistent with Account Fee Rate, both must be BUSINESS, or both must be ‘PERSONAL’.  Such a business rule would be critical on an Anti Money Laundering Programme, where you must apply different alert rules to personal and business accounts.

In some organisations, there is no ‘Master’ repository of business rules.  Business rules are not easily researched, not formally documented, and not subject to change control.

Impact: Projects dependent on existing data must research business rules governing that data from first principles, and face the risk of not finding them, or finding inconsistent business rules.  This leads to duplication of effort, project delays, and the risk of making incorrect business decisions based on incorrect business rules (e.g. generating False Anti Money Laundering Alerts on accounts you treat as PERSONAL, when in fact they are BUSINESS.)

Solution:
Agree and implement the following Policies:

  1. Overall ownership for business rules governing data within the Enterprise lies with the CIO.
  2. Ownership for business rules within each Business Unit lies with the CIO and the head of the Business Unit.
  3. Business rules must be formally documented and subject to change control (Enterprise-wide, and Business Unit specific).
  4. The CIO must appoint a person (TitleX) with responsibility for Enterprise wide business rules.
  5. TitleX is responsible for the definition and maintenance of Enterprise-wide business rules, in consultation with business units.
  6. TitleX must provide a single point of contact to handle requests for business rule details.

Your experience:
Have you faced the above issue in your organisation, or while working with clients?  What did you do to resolve it?  Please share your experience by posting a comment – Thank you – Ken.

Common Enterprise wide Data Governance Issues #9: Data Migration and ETL projects are Metadata driven

This post is one of a series dealing with common Enterprise Wide Data Governance Issues.  Assess the status of this issue in your Enterprise by clicking here: Data Governance Issue Assessment Process

Too often, Data Migration and ETL projects are built on the basis of Metadata, without measuring what is actually contained in the source data fields.  This happens when the IT function build data ‘pipes’ on the basis of what the metadata says the source fields should contain, and don’t perform data content analysis, or data profiling, to find out what the source fields actually contain.

Impact:
The IT function turn the  ‘tap’ on, the data flows through the ‘pipes’ and the business express surprise, followed by denial, when expectations cannot be met due to data quality issues.  This is known as the ‘Load and Explode’ approach to data.

Solution:
To prevent ‘Load and Explode’ impacting the success of your data dependent projects, agree and apply the following policy:

Before building, or purchasing a system that is dependent on existing data, projects must complete the following process:

  1. Define what data is required.
  2. Define the quality requirements of the required data.
  3. Identify the source of the required data.
  4. Specify the data quality metrics to be captured.
  5. Measure the quality of the available source data.
  6. Understand the implications of the quality of available source data for the proposed system.
  7. If necessary, and if feasible, implement data quality improvement measures to raise the quality to the required level.
  8. Worst case – if the facts tell you data quality is too low and cannot be improved – Cancel the project and save yourself a ton of money!

Your experience:
Have you faced the above issue in your organisation, or while working with clients?  What did you do to resolve it?  Please share your experience by posting a comment – Thank you – Ken.

Common Enterprise Wide Data Governance Issues: #8 Accessibility of Data is poor

This post is one of a series dealing with common Enterprise Wide Data Governance Issues.  Assess the status of this issue in your Enterprise by clicking here: Data Governance Issue Assessment Process

Accessibility of Data is poor.

  • Business users find it difficult to identify who to ask regarding where to find required data (information).
    e.g. Compliance Units regularly receive regulatory requests for new details.  In some enterprises it is unclear who the Compliance Unit should ask regarding locating the required data (information).
  • Busines users experience difficulty and delays in locating required data (information).
  • Miscommunication between the business and IT lead to IT misinterpreting the data requirement and identifying inappropriate data.

Impact: Projects dependent on existing data must locate the data from first principles, and face the risk of not finding the data, or identifying inappropriate sources.

Solution:
Agree and implement the following policies:

  1. Overall ownership for data within the Enterprise lies with the CIO.
  2. Ownership for data within each Business Unit lies with the CIO and the head of the Business Unit.
  3. The CIO and head of each business unit must appoint a person with responsibility for the provision of data from that business unit, to those who require it.  This person is also responsible for the measurement and maintenance of data quality within the business unit.
  4. The CIO and head of each business unit must appoint a single point of contact to handle requests for data / information.

Your experience:
Have you faced the above issue in your organisation, or while working with clients?  What did you do to resolve it?  Please share your experience by posting a comment – Thank you – Ken.

Common Enterprise Wide Data Governance Issues: #7 No SLAs defined for required quality of critical data

This post is one of a series dealing with common Enterprise Wide Data Governance Issues.  Assess the status of this issue in your Enterprise by clicking here:  Data Governance Issue Assessment Process

In some organisations there is a platitute that states: ‘The Business is responsible for the quality of the data’, but…

  • There are no SLAs defined for the required quality of critical data (Master Data)
  • There is no measurement performed of the actual quality of the data
  • The are no processes available to “The Business” to enable them to measure the quality of the data

Impact: Multiple enterprise wide data quality issues.

Solution:
Agree and implement the following policies:

  1. “The business” must be provided with Standard Enterprise wide data quality measurement processes and tools
  2. Business units must regularly measure the quality of critical data, using standard Enterprise wide processes and tools, and must agree SLAs with the users of the data defining the target quality level.
  3. Where necessary, business units must implement data quality improvement measures to meet the quality defined in the agreed SLA.

Your experience:
Have you faced the above issue in your organisation, or while working with clients?  What did you do to resolve it?  Please share your experience by posting a comment – Thank you – Ken.

Common Enterprise Wide Data Governance Issues: #6 No Enterprise-Wide Data Quality Measurement of existing Data Content

This post is one of a series dealing with common Enterprise Wide Data Governance Issues.  Assess the status of this issue in your Enterprise by clicking here:  Data Governance Issue Assessment Process

Impact: ‘If you do not measure, you cannot manage.’

There is a risk that either Data Quality is not measured at all, or is measured on a piece-meal localised basis, using non-standard processes and producing questionable results.

Solution:
Agree and implement the following policy:

  1. The quality of critical data (Master Data), in critical databases must be measured regularly, using an Enterprise wide standard set of processes and tools, producing a standard set of quality metrics covering a standard set of quality dimensions.
  2. Data quality SLAs must be agreed and implemented between the owners of the data, and the end users of the data.

Your experience:
Have you faced the above issue in your organisation, or while working with clients?  What did you do to resolve it?  Please share your experience by posting a comment – Thank you – Ken.

Common Enterprise Wide Data Governance Issues: #2 The quality of data entered by front-end staff is not as high as desired

This post is one of a series dealing with common Enterprise Wide Data Governance Issues.  Assess the status of this issue in your Enterprise by clicking here:  Data Governance Issue Assessment Process

Causes:

  1. Front-end staff are put under pressure to ‘sell business’ and sort out details later.
  2. Front-end staff have no incentive to enter high-quality data.
  3. Front-end staff are unaware of the downstream impact of poor data.
  4. Little realisation on the part of front-end staff that data entered at account opening may be re-used repeatedly for 30+years.
  5. Front-end systems may lack data validation software.
  6. Front-end systems may not enforce business rules between related fields.
  7. Reluctance to add data validation to legacy Front-end systems

Impact:

  • Incomplete or poor quality ‘informational’ or ‘Master data’ captured at data entry.
  • Risk that the enterprise and/or individual business units will fail to comply with regulatory requirements (e.g. the requirement to ‘Know your customer’).

Solution:
Agree and implement the following policies:

  1. Data owners must define the business rules to be applied to each data item at data entry time.
  2. Data owners must agree Data Quality requirements with the end users of the data, and the owners of the data entry processes.
  3. Data owners must agreee Data Quality Service Level Agreements with the end users of the data, and the owners of the data entry processes.
  4. Data Quality measurement must be implemented at each point of data entry to measure the quality of data being entered, and support the implementation of data quality Service Level Agreements.
  5. Front-end staff must be educated on the importance of data quality
  6. Front-end staff must be supported in taking the time and effort to “get it right first time”

Your experience:
Have you faced the above issue in your organisation, or while working with clients?  What did you do to resolve it?  Please share your experience by posting a comment – Thank you – Ken.