Common Enterprise wide Data Governance Issues – #12. No Enterprise wide Data Dictionary.

This post is one of a series dealing with common Enterprise Wide Data Governance Issues.    Assess the status of this issue in your Enterprise by clicking here:  Data Governance Issue Assessment Process

Anyone know what this means?

An excellent series of blog posts from Phil Wright (Balanced approach to scoring data quality) prompted me to restart this series.  Phil tells us that in his organisation, “a large amount of time and effort has been applied to ensure that the business community has a definitive business glossary, containing all the terminology and business rules that they use within their reporting and business processes. This has been published, and highly praised, throughout the organisation.” I wish other organisations were like Phil’s.

Not only do some organisations lack “a definitive business glossary” as Phil describes above, complete with business rules….
Some organisations have no Enterprise wide Data Dictionary.  What is worse – there is no appreciation within senior management of the need for an Enterprise wide Data Dictionary (and therefore no budget to develop one).

Impact(s):

  • No business definition, or contradictory business definitions of the intended content of critical fields.
  • There is an over dependence on a small number of staff with detailed knowledge of some databases.
  • Incorrect or non-ideal sources of required data are identified – because the source of required data is determined by personnel with expertise in specific systems only.
  • New projects, dependent on existing data, are left ‘flying blind’.  The impact is similar to landing in a foreign city, with no map and not speaking the language.
  • Repeated re-invention of the wheel, duplication of work, with associated costs.

Solution:

CIO to define and implement the following Policy:  (in addition to the policies listed for Data Governance Issue #10):

  • An Enterprise wide Data Dictionary will be developed covering critical Enterprise wide data, in accordance with industry best practice.

Does your organisation have an “Enterprise wide Data Dictionary” – if so, how did you achieve it?  If not, how do new projects that depend on existing data begin the process of locating that data?  Please share your experience.

14 thoughts on “Common Enterprise wide Data Governance Issues – #12. No Enterprise wide Data Dictionary.

  1. Hi Ken,

    Good post.

    I think one of the hardest things to do in this type of exercise is getting the business to talk to one another, and then, actually getting them to agree with one another.

    This isn’t the type of exercise that can be undertaken over E-Mail. You need to get the right people in the same room, and discuss.

    Another challenge is how to store multiple definitions of the same terms. In some cases we will never escape multiple definitions, so ensuring that they are labelled in a way which can identify in which context they should be applied is critical.

    ie. What is an active customer? All customers who have an account with us, or all customers with activity on their account in the past 180 days?

    Having the ability to make the dictionary collaborative is useful – if someone doesn’t agree with a definition they could challenge it. If a definition is missing, someone could add it.

    Like everything in the world of Data Quality, a data dictionary isn’t a one-off exercise. It needs administrative care just like any source of information, or else it risks growing out of control, or worse, obsolete.

  2. Hi Phil,

    Thanks for your comment.

    Your “Active Customer” example is excellent. In the context of an Anti-Money Laundering system, an “Active Customer” would be a customer with activity on their account in the past “n” days. In fact – accounts that become active, after a period of inactivity are considered a Money Laundering risk, since they may have been “taken over”.

    The marketing dept. on the other hand, would probably view all customers with an account with us, as “active”.

    Getting different business areas to understand, and agree DIFFERENT definitions for the same term is a challenge – but it must be done.

    Rgds Ken

  3. Great post Ken,

    As you and Phil have said, a data dictionary isn’t a one off exercise.

    But here’s a question, who owns the data dictionary, and the subsequent maintenance thereof?

    Regards

    Charles

  4. Ken

    Again, I have to be the second Phil to the dance. I really need to get my new cartoon alter ego to get his ass in gear…

    http://www.philsimonsystems.com/tag/travails-of-a-technology-consultant/

    But I digress

    Great post. I have never seen any of the 50 or so organizations that I’ve visited/worked at have anything resembling a data dictionary. The best that I’ve seen is that the people will rely on the vendor’s data dictionary in the apps.

    While this is certainly better than nothing, it’s generic and not tailored to the organization.

    You really hit the nail on the head with institutional knowledge being in the heads of fleeting employees, not shared in any repository. Perhaps Enterprise 2.0 collaborative tools will help in this regard.

    Of course, people have to use them. What’s more, they will need to be accountable if they don’t.

    Great post.

  5. Charles, good question. The job’s yours if you want it? 😉

    At least 2 of the telecoms organisations I’ve worked for had dedicated ‘Data Management’/’Data Stewardship’ Teams, who had the responsibility of maintaining a data dictionary. It was actually written into their objectives.

    However, at one of those organisations, I know that certain areas of the business were suggesting that the dictionary wasn’t up to date.

    But whose fault is this? Any new/altering definition needs to be communicated to the correct parties. But how?

    * Via a network for data stewards?
    * By ensuring that all definitions raised in new projects are cross-referenced, or added to the central business dictionary?

    It certainly is an administrative challenge, and very often a cultural challenge too.

  6. Phil,

    Let me think about that offer, ummmm, errrrr, no thanks 🙂

    It’s a thankless task, I’ve implemented ‘ownership’ of the data dictionary in a number of ways through the years, and as you mention, wherever it sits the administrative challenge is ever present, and never small.

    The best success I have had is via a true Data Governance Board with senior staff members involved, people who can debate and make decisions on data there and then. General day to day admin was managed via the central data quality team.

    Data Governance by committee I suppose …

  7. Thank you Charles, Phil and Phil (!)

    To summarise:
    1. Data Dictionary / Business Glossary is critical.
    2. Most organisations don’t have one.
    3. Having one is not enough – it must be owned, maintained and readily accessible as the living, up to date “source of information”.
    4. Appointing correct owner, and decision makers is critical.

    The above summarises the Policy, People and Process items.

    I agree with Phil Simon, Enterprise 2.0 collaborative tools may help…
    E.G. WikiBusinessGlossary – where business users may find the details they need, or request a definition for a new term.

    Ken

  8. Great article and follow up discussion, here are some thoughts based on my personal experience;

    1) initiation

    getting business/IT/operations together is sometimes like all the planets converging, you need to grasp the opportunity as it may not come along again for a while… a data dictionary is a great tool for synchronising definitions and rules between groups and the various levels of detail should then inform your software/data lifecycle management

    2) ongoing

    much more difficult is getting the data dictionary to influence and reflect the IT/Ops landscape and conversely for IT/Ops changes to feedback into the Data Dictionary, so that business has an uptodate view of reference information and where it is exposed and/or confidence (or lack of) in the reality of the information. Robust and transparent change management processes are thus critical… as is the ability of your Data Dictionary to react to this change in a controlled way.

    http://www.twitter.com/jwmuk

    • Jim,

      You have highlighted two processes that are “time dependent”. The problem, as you point out, is that the second process “ongoing maintenance of the data dictionary” is seldom performed.

      It reminds me of my own experience…

      I have helped a number of clients with classic “Regulatory Compliance”, “Must be done NOW” projects.

      Usually, my clients have already selected a “best in breed” solution. The solution provides the regulator with all the required answers – Simple!

      All the client needs to do is “populate the database supplied by the vendor” !! That’s usually when I get a call – “Ken, can you help us locate the data we require…”?

      These “tactical solutions” invariably create yet another replica of customer and account data. Finding the Master source(s) is always a challenge. Unearthing the business rules associated with the master data usually proves even more difficult.

      The “Initiation” – getting business/IT/operations together, is like pulling teeth – with lots of miscommunication between the Business and IT, due to the different language spoken on each planet.

      I’ve been fortunate enough to have successfully helped my clients complete their “tactical projects”. I always highlight the need for “Ongoing change management” after the project to ensure the new replica, (and the other replicas we found in the process), are kept in sync with the Master. I also point out that if the “Ongoing Change Management” was in place when we started – we would have completed the project more quickly and more cost effectively.

      Ongoing work requires ongoing budget. Tactical projects are assigned a limited budget. When the budget runs out, the work stops.

      I suspect I’ll get similar calls in future – what’s the next big Regulatory requirement ????

      Ken

  9. Excellent post and really interesting comments you’ve generated here Ken.

    I’ve seldom seen these done correctly but I agree the only way is to have a committed board driving standards and auditing the dictionary regularly.

    It also needs to be accessible, one of my biggest issues that we often make the updating of these assets so damn hard.

    I think ultimately though without some serious funding and maintenance, any slight inaccuracy or inconsistency can cause a lack of trust, even if 90% of the content is spot on.

    But where there is pain there is gain, if someone could create a simple, affordable, accessible platform they would have a killer solution, anyone have any ideas?

    (Or some VC capital…)

  10. Great post and comments.

    I have recently initiated the creation of an enterprise wide data dictionary – see http://michaelbaylon.wordpress.com/2010/01/27/what-do-you-mean/ – for similar reasons outlined in the original blog.

    A bottom up approach was taken – gradually building it up over the last year.

    As stated earlier – there is little appreciation for the need for one and even if people in the business do think it is a good idea – they don’t normally see it as a high priority.

    Part of the process over the last year has been to record examples of how we have/are using it to solve real business problems and communicating these as often as possible.

    This have bought some success and its use – eg identifying data owners and updating terms/definitions – is about to be included as part of the project life cycle.

    It is hoped that this will start to solve some of the issues around “ongoing maintenance of the data dictionary”.

    But the ideal would be for business users to maintain it. Like the idea of ‘WikiBusinessGlossary’ – trying to make it as easy as possible for users to update it.

  11. Great post and discussions,

    I believe many of us who are confronted with the impact of a lack of shared definitions and rules know how badly this is needed.

    Collibra is focussing on exactly this. Creating a business friendly glossary with definitions, examples, but also business facts and rules using OMG’s new Semantics for Business Vocabulary and business Rules (SBVR) standard.
    Because we can add the necessary formality, organisations can automatically generate technical data models (UML, XSD, …) out of the glossary. This provides a closed loop governance and transparency between the business and the technical implementations, which is in our view extremely important to achieve the required business benefits.

    We also take into account different definitions by different communities of interest (the extremely important notion of context someone referred to).

    As a front-end, we use the wiki-paradigm, which makes it extremely easy to involve all stakeholders, business and IT.

  12. Again, great post with discussions.

    I think the idea of using a Wiki-based Data Dictionary is one worth exploring. I think it provides a way of allowing easier access and usability for all.

    At my current organisation, there has been a Data Dictionary for one system that has been implemented on a wiki. So far, I think it is pretty good, but those who use it claim it is not kept up to date.

    So this comes back to the question: Who owns (and therefore should update) the DD? Well surely, the point of a wiki is that the users themselves are responsible for updating it? It could then fall to a team of Data Stewards (or similar job title) to be the administrators of this wiki, alerted to new changes made and checking for appropriateness etc. This way the onus is placed more on the users and leaves less of a daunting task for Data Stewards, and ultimately should lead to a more accurate DD.

    I’d love to hear any follow-up thoughts on this.

    My second question is more generic. How do we begin to populate these Data Dictionaries?
    There are technologies out there which will map a system (an oracle DB for example) and provide you with table names, column names, datatypes etc. If these can be exported to XML, then can we not convert the XML to our chosen DD technology? Sure, it won’t fill in everything we need, especially when it comes to more context-specific business rules, but that is where the humans come in!

    Again, I’d be interested in feedback.

    Lastly, Dylan – If you can take this idea on Dragons Den, I’m with you! 🙂

  13. Michael, Felix and Kevin,

    Thanks for your comments – sorry for delay in replying.

    To summarise, there is a clear need for an enterprise wide data dictionary. Having one is not enough, it must be trusted by the users.

    The concept of a wiki-based DD appears popular – but how would one perform the initial population? Who would maintain it? Who would QA it?

    Challenges and opportunities…

    Thanks again for your input

    Ken

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s