Opportunity to apply lessons learnt in my new job

This week I started a new job as Head of Customer Information at Bank of Ireland in Dublin. I am excited at the prospect of applying the lessons I have learnt for the benefit of our customers.

I would like to take this opportunity to thank my fellow data management professionals worldwide for generously sharing their experience with me. I started to write this blog in 2009. My objective was to “Share my experience and seek to learn from the experience of others”. I have certainly learnt from the experience of others, and I hope to continue to do so.

The opinions I express on this blog will continue to be my own. I look forward to continuing to hear yours.

How to deal with Gobbledygook requirements

In my last post I had a bit of a rant about the Gobbledygook “eligibility requirements” provided by the UK Financial Services Compensation Scheme.

The reality is that business requirements can come from many places, they are often vague, and often overly complex.  They are often imposed on you from outside, as in the case of regulatory requirements, like the UK regulatory requirement to deliver a Single Customer View.

So… life can be tough – you have to get on with it, and deal with “less than perfect” requirements.

Well defined requirements are clear, measurable and testable.  You cannot expect business experts to be expert in defining requirements. As a Data Quality professional, one of your roles is to work with business experts to clarify and simplify their requirements.

Let us look at the “eligibility requirements” provided by the UK Financial Services Compensation Scheme

In essence, some customer types are eligible for compensation, while others are not.  You must use your “parsing skills” to parse the overly complex rules – to “sort the apples from the oranges” so to speak.   Start by listing unique customer types, which include:

  • Sole Trader
  • Credit Union
  • Collective Investment Scheme
  • Trustee of a Collective Investment Scheme
  • Operator of a Collective Investment Scheme

Having done that, you can begin the task of finding out whether you can currently identify these customer types within your data.

The above is just a starting point, I hope it helps.

Feedback welcome, as always.

What does complete appropriate and accurate mean?

Welcome to part 2 of Solvency II Standards for Data Quality – common sense standards for all businesses.

The Solvency II Standards for Data Quality run to 22 pages and provide an excellent substitute to counting sheep if you suffer from insomnia. They are published by The Committee of European Insurance and Occupational Pensions Supervisors (CEIOPS) (now renamed as EIOPA).

Solvency II Data Quality Standards – not as page turning as a Dan Brown novel

I accept that Data Quality Standards cannot aspire to be as page turning as a Dan Brown novel – but plainer English would help.

Anyway – enough  complaining.  As mentioned in part 1, the standards require insurance companies to provide evidence that their Solvency II submissions are based on data that is “as complete, appropriate, and accurate as possible”.  In this post, I will explore what the regulator means by “complete”, “appropriate” and “accurate”.  I will look at the terms in the context of data quality for Solvency II, and will highlight how the same common sense standards apply to all organisations.

APPROPRIATE: “Data is considered appropriate if it is suitable for the intended purpose” (page 19, paragraph 3.62).

Insurance companies must ensure they can provide for insurance claims. Hence, to be “appropriate”, the data must relate to the risks covered, and the value of the capital they have to cover potential claims.  Insurance industry knowledge is required to identify the “appropriate” data, just as Auto Industry knowledge is required to identify data “appropriate” to the Auto industry etc.

COMPLETE: (This one is pretty heavy, but I will include it verbatim, and then seek to simplify – all comments, contributions and dissenting opinions welcome) (page 19, paragraph 3.64)

“Data is considered to be complete if:

  • it allows for the recognition of all the main homogeneous risk groups within the liability portfolio;
  • it has sufficient granularity to allow for the identification of trends and to the full understanding of the behaviour of the underlying risks; and
  • if sufficient historical information is available.”

As I see it, there must be enough data, at a low enough level of detail, to provide a realistic picture of the main types of risks covered. Enough Historical data is also required, since history of past claims provides a basis for estimating the scale of future claims.

As with the term “Appropriate”,  I believe that Insurance industry knowledge is required to identify the data required to ensure that data is “complete”.

ACCURATE: I believe this one is “pure common sense”, and applies to all organisations, across all industries. (page 19, paragraph 3.66)

Data is considered accurate if:

  • it is free from material mistakes, errors and omissions;
  • the recording of information is adequate, performed in a timely manner and is kept consistent across time;
  • a high level of confidence is placed on the data; and
  • the undertaking must be able to demonstrate that it recognises the data set as credible by using it throughout the undertakings operations and decision-making processes.

Update – In October 2013, following an 18 month consultative process, DAMA UK published a white paper explaining 6 primary data quality dimensions.

1. Completeness
2. Uniqueness
3. Timeliness
4. Validity
5. Accuracy
6. Consistency

For more details see my blog post, Major step forward in Data Quality Measurement


Semantic web and data quality

I have been itching to write this post since reading and listening to Phil Simon’s excellent blog post and podcast – Technology Today, #20: David Siegel and The Semantic Web .  If you havn’t listened to Phil’s interview with David Siegel – I recommend you do so now.   I have listened twice so far, and expect to listen many more times.

I have previously discussed Plug and Play Data – The future for Data Quality and I believe that David’s ideas about the “Pull model” and “the power of pull” will help deliver Plug and Play data.

For example, David explains “Semantic” as “Unambiguous”.   As you know, data entry validation against unambiguous business rules can greatly improve data quality.  The semantic web will enable us to embed our clear unambiguous business rules with the data the rules apply to – WOW ! Bring it on !

How to deliver a Single Customer View

How to deliver a Single Customer View

How to cost effectively deliver a Single Customer View

Many have tried, and many have failed to deliver a “Single Customer View”.  Well now it’s a regulatory requirement – at least for UK Deposit Takers (Banks, Building Societies, etc.).

The requirement to deliver a Single Customer View of eligible deposit holders indirectly affects every man, woman and child in the UK.  Their deposits, large or small, are covered by the UK Deposit Guarantee Scheme.  This scheme played a key role in maintaining confidence in the banking system during the dark days of the world financial crisis.

UK Deposit Takers must not only deliver the required Single Customer View data, they must provide clear evidence of the data quality processes and controls they use to deliver and verify the SCV data.

The deadline for compliance is challenging.  Plans must be submitted to the regulator by July 2010, and the SCV must be built and verified by Jan 2011.

To help UK Deposit Takers, I have written an E-book explaining how to cost effectively deliver a Single Customer View.  You may download this free from the Dataqualitypro website:

While the document specifically addresses the UK Financial Services Requirement for a Single Customer View, the process steps will help anyone planning a major data migration / data population project.

If you are in any doubt about the need for good data quality management processes to deliver any new system (e.g. Single Customer View, Solvency II, etc.), read the excellent Phil Simon interview on Dataqualitypro about why new systems fail.

Show me your Data Quality

Recently Marty Moseley explored how much data governance you should have in a thought provoking post called how taxing is your data governance.

I added the following comment “I agree with you – lightweight governance is the way to go – as you say “just formalized enough” Data Governance  framework, creating “good enough” deliverables to record your decisions, alternatives, precedents, drivers, policies, procedures/processes, business rules, enforcements and metrics – and find them later when you need to invariably make changes – OR WHEN THE REGULATOR asks to see your audit trail.

The fact is that regulators increasingly require evidence of the data quality of the underlying data on which you base your regulatory submissions.  Not just that they require evidence of your data quality management processes, i.e. your Data Governance.

The law now requires evidence of your data quality

For example, in the UK, Deposit Holders are required to build a Single Customer View (SCV) of depositors (UK FSA Single Customer View requirement)  They must not only build an SCV, the regulation states “We… would need to verify that the SCV information being collated and stored by a deposit taker was adequate and fit for purpose”.   Consultation Paper CP 09/3 about the SCV states “As part of their data cleansing, deposit-taking firms would be required to ensure the existence, completeness and accuracy of all data required for each depositor to facilitate fast payout”

A second example affects Insurance companies providing insurance in Europe (this also affects US Insurers operating in Europe).  EU Directive 2009/138/EC (Solvency II) – Article 82 states: “Member States shall ensure that insurance and reinsurance undertakings have internal processes and procedures in place to ensure the appropriateness, completeness and accuracy of the data used in the calculation of their technical provisions.” In other words – the regulator in each member state is required to review and approve the Data Governance processes of insurers.

Failure to comply with Regulatory requirements can prove expensive. Ask Standard Life.  It cost them over £100 million.  I presented details of the Standard Life case at the Information and Data Quality Seminar Series 2010 – Dublin. (For more details, see slides 11 and 12 in my presentation “Achieving Regulatory Compliance – The devil is in the data“.)

Shortly, working together with DataqualityPro, I will publish an E-Book on how to cost-effectively satisfy the (UK FSA Single Customer View requirement).  UK Deposit Holders must submit their SCV implementation plans to the FSA by the end of July 2010, and must complete their SCV by Jan 2011.  Time is running short.

Are you aware of other laws requiring evidence of Data Governance and Data Quality? If so, please share them.

Common Enterprise wide Data Governance Issues – #13. Islands of data.

Islands of data across the Enterprise.

This post is one of a series dealing with common Enterprise Wide Data Governance Issues.    Assess the status of this issue in your Enterprise by clicking here:  Data Governance Issue Assessment Process

In many large organisations, tactical localised projects over many years has resulted in multiple databases of similar but inconsistent and unlinked data (e.g. multiple customer databases).

Historically, business area owners have been reluctant to share “their data” with other business areas – for fear of data corruption, or occassionally for reasons of “empire building”.

Impact(s):

– Information Management nightmare

– Multiple versions of “the truth’.

– No ‘single view of customer’.

– Difficulty in meeting regulatory requirements in the area of ‘Know Your Customer’.

– …

Solution:

CIO to define and implement the following Policy:

  1. New data stores must only be created in consultation with the Enterprise Data Architect, taking into account the target Enterprise Data Model.
  2. Measures are required to replace local islands of operational data with Enterprise wide operational databases, and to implement an Enterprise wide data warehouse.

Does your organisation have “islands of data” – if so, what problems do they cause you?

If not, how did you get rid of them?  Did you build “bridges”?  Do your data islands feed into a central “Information Warehouse”.   Please share your experience.