Business Rules Case Study Part I

I would like to start a discussion about Business Rules.  I hope you will join in.  Over a series of posts I plan to explore questions like:

  1. Why are Business Rules necessary?
  2. What exactly is a Business Rule?
  3. What should happen if the data fails a Business Rule?
  4. What controls should you have in place to manage Business Rules?
  5. Where should you look for Business Rules (if your Enterprise has no Master Business Rules Repository)

I will use a case study from an Anti Money Laundering (AML) programme.

In this AML programme, the client selected a “Best in breed AML vendor solution”.   The vendor specified the data required, and the client was responsible for locating the data to populate the new AML repository, and for the quality of the data entered in the repository.

Why are Business Rules necessary?

A standard AML business requirement is the requirement to monitor “Minor Accounts” (accounts held by customers under 18 years of age) for ‘unusual transaction activity’.  This high level requirement would result in a number of more specific business requirements, such as:

“Generate an AML alert when the total value of Cash lodged in a month, to an account held by a minor, exceeds a predefined amount, say EUR5000”

Having  agreed the above business requirement, the vendor asked the client to provide the Business Rule for identifying a ‘Minor Account’.

So:

1. Why are Business Rules necessary?
Business rules are required to distinguish between product types, customer types, car parts etc. etc.  AML systems require business rules in order to apply different alert rules to different account holder types.

AML business staff are AML experts, not business rules experts.  It was unclear who owned the data and it took a long time for the IT department to research the business rule(s) for the vendor.  Q:  How do business users in your enterprise get details of Business Rules?  Do your business users find it difficult to access the data they require?

Let us suppose the Business Rule supplied to the vendor was:
A minor account may be identified as follows:
1. Account Type: Personal
2. Account SubType:  Minor
3. Customer Age:  Less than 18

The age check was required to manage the risk that an account opened when a customer was a Minor was not converted to a Standard Personal account when the customer reached his/her 18th birthday.

So:

2. What exactly is a Business Rule?

A Business rule provides critical details about data, including the ‘business’ name of the field, the business purpose of the field, the values it may hold, the business meaning of each value, and interdependencies with other data.  Let’s explore this a little further:

  1. Business name of the data field(s):
    In the above example, three data fields are used in the Business Rule:
    ‘Account Type’, ‘Account Subtype,’ and ‘Age’ (probably determined from Date of Birth).’
  2. Business purpose of the data field:
    e.g. ‘Account SubType’ is used to identify different account types, such as ‘Minor’, ‘Mature years’ etc.
  3. Permitted values:
    e.g. Permitted values for Account Subtype are 101 to 199.
  4. Business meaning of each permitted value:
    e.g. ‘Account SubType’ value 101 means Minor Account
  5. Interdependencies with other data:
    e.g. ‘Account SubType’ depends on ‘Account Type’
    ‘Account SubType’ value 101 means Minor Account, when Account Type is ‘Personal’
  6. Field precedence:
    This defines the order in which the fields should be interrogated
    e.g.  First check Account Type, then Account Sub Type

The AML vendor configured the AML tool to apply the “MINOR” rule when Account Type was personal, Account SubType =101 (Minor), and Customer Age less than 18.

During testing, few alerts were generated on Minor accounts.  From an AML business perspective, the less alerts generated the better, since the workload for the AML staff is dictated by the number of alerts generated.

The AML business area was pleased with the low number of alerts, and the vendor was pleased that the alert worked ‘as specified’.

However, it was common knowledge that Date of Birth was not populated 100% of the time, so what was happening when there was no Date of Birth present?  There was no culture of  data quality measurement in the Enterprise, and no facilities for data profiling. Custom built SQL queries against the new AML repository identified multiple instances in which the actual data failed to conform to the Business Rules.

So:

3. What should happen if the data fails a Business Rule?
In our AML example, what do you think should happen when:
a) Account Subtype is ‘101’ indicating a MINOR account, but the customer is aged over 18?
b) Account Subtype is ‘101’ indicating a MINOR account, but date of birth is not populated for this customer?

Business Rules define what data fields “should” contain.  On this AML programme, as in all real world data programmes, the actual data content did not always match what was expected.

This only became apparent as a result of custom built data profiling.  Based on the actual content of the data, the AML business area had to ask the vendor to implement Exception Rules to handle the non-conforming data.  In an ideal world, the data would have been corrected.  In the real world of “achieve compliance by a given date, or face a regulatory fine”, workarounds are quite normal. 

So – what are Exception Rules?
Exception rules define what must happen when an account contains data that fails to comply with a business rule.

This post is already far longer than I had planned – I hope it hasn’t bored you to tears.
In my next post, I will explore:

Please share your experience by posting a comment – Thank you.

10 thoughts on “Business Rules Case Study Part I

  1. Ken,

    Excellent discussion – I really like the use of a detailed case study.

    Although I have no AML experience to share, I can share my experiences with alerts and exceptions.

    On many data migration and data integration projects, I have seen ETL programmers write simple data validation routines to check for source data values that do not conform to a high level business rule.

    Typically, these “exceptions” would at least be logged to an error report and an alert would be sent to the business team (typically via an automated email message).

    What varied by project was whether the record containing the exception would still be loaded into the target table or be “suspended” in an error table.

    Many of these projects used very simple validation checks and therefore few exceptions were actually reported, which pleased the business team – same as your case study example.

    However, as the technical lead, one problem that I would often find during a code review with the ETL programmer is something like the following pseudo-code (borrowing from your example):

    IF (Account Type = ‘Personal’ AND Account Subtype = 101 AND (ISNOTNULL(Age) AND Age > 18)) THEN Report_Exception

    I would ask the programmer: “what happens when Age is NULL?” and the response typically was “technically, there isn’t a business rule defined for that scenario.”

    The other problem that I would often encounter was getting the business to take action when exceptions were reported.

    It was as if they wanted validation checks to be performed, but they never wanted them to actually find any exceptions because “we don’t have a business process for correcting exceptions.”

    Sadly, the most common problem was that no business rules were defined at all and the data would be blindly migrated or integrated without even at least some superficial validation checks.

    I am looking forward to the rest of the series.

    Best Regards…

    Jim

    • Jim,

      Thanks for joining the discussion. You have raised some excellent points, and opened my eyes to additional topics I hope to cover in this series. I believe your last paragraph, regarding ‘Blind Migrations’ is particularly important for us Data Quality professionals. I refer to this as the ‘load and explode’ migration method, in Common Enterprise Data Issue #9. The challenge for us is to prevent this situation arising, by convincing “The business” of the need for ongoing Data Quality Measurement. Hence when the time comes to migrate the data (as invariably it will), most of the Data Content Quality is understood, and there is only a need to perform additional validation that may be specific to the Migration in question.

      Keep the comments coming – thank you,

      Ken

  2. Thanks for opening this discussion Ken.

    Often I find it useful to divide business rules into 2 pots:

    • External rules that are defined outside your organisation – mostly laws and other regulations you must follow when doing business in a given country (or group of countries like the EU).
    • Internal rules that are defined by and for your business alone – made to make your business competitive.

    This division is similar to the division of data into:

    • External Reference Data defined as data initially born and maintained outside your organisation.
    • Internal Master Data and Transaction Data born and maintained inside your organisation for your specific purposes – governed by your specific business rules.

    More on Master Data etc. here.

    We already know about ready made software solutions holding many external business rules sometimes supported by external reference data.

    My guess is that we will see increased number of solutions holding increased number of external business rules and default internal business rules e.g. industry specific – supported by increased number of external reference data.

    Cloud computing will contribute a lot to that development and also SOA (Service Oriented Architecture) components will help organisations deploy rules consistently and implement all the external business rules that are exactly the same for every business in a given part of the world.

    All the best

    Henrik

  3. Henrik, (and Jim earlier),

    This is exactly what I was hoping for. People joining in to expand our shared knowledge.

    I like your concept of dividing business rules into External and Internal rules. They are different (just like apples and oranges). External business rules are more suited to standardisation, since they are by definition ‘global’. The more pre-defined standards that businesses can tap into, the easier life becomes.

    I suspect that dealing with the internal business rules will continue to present challenges into the future.

    A particular challenge will arise in situations where internal bespoke rules should be replaced with industry standard external rules. The bespoke internal rules may have been developed many years ago, and may be referenced by many applications across an Enterprise.

    Any suggestions on how this could be achieved?

    Rgds Ken

  4. Great topic Ken, thanks for that.

    What do you think about a Business Rule holding some indication of the business impact associated with a data field? For example we might have 30 fields all of which have valid purpose but 7 of them have high importance because if the quality is low on those then there is potential for the company to lose financially (or in other ways). If our exception reporting is working it would be nice to prioritise the fixes. If we want greater granularity then the exception rules could also hold an importance rating.

    Where workarounds are employed, how often are they implemented by relaxing an exception rule? Is there ever a process in place which re-instates the rule after the source fix?

    Regards,

    Phil

  5. Hi Phil,

    Thanks for joining the discussion.

    In my experience, too few organisations maintain business rules for their data fields. This means that downstream projects that require the data must research the business rules from first principles.

    I have faced this situation too many times. Given time constraints etc. the focus has to be on the most critical data fields. Thus in my experience, we identified the required data fields, then ranked their criticality as High Medium and Low, and focussed our efforts on the Highly critical ones.

    So to answer the first part of your question – should a business rule hold some indication of the business impact associated with a data field? Absolutely.

    Your second question suggests that the source (data) will be fixed. I wish I worked in your organisation!

    Much of my experience has been on migrations, and conversions. The source data has often been in place for many years (or decades). The objective has been to populate a new repository (e.g. for a new AML system, or BASEL II system) or to convert data ‘in situ’ (for Euro Changeover). Cleaning data is typically outside the scope of such programmes. (We would highlight the issue(s), and recommend that the data be cleansed). TWO workarounds were typically required – the first in the data population process, to allow the record containing the ‘bad data field’ into the new system. The second workaround in the new system, to recognise the fact that the data contains anomolies, and to implement “exceptional business rules” to deal with the anomolies.

    In an ideal world, two actions would occur:
    1. Business rule validation would be implemented at point of data entry in the original source system (this is becoming easier with ‘Cloud based’ verification from a number of vendors). This is required to ‘stop the rot’, and prevent ongoing data errors.

    2. A Data Cleansing programme is required to correct the “existing bad data”. Assuming we are talking about customer information, this is more difficult, since ideally one would contact the customer to verify and correct customer details. Hugely labour intensive, and impractical. The goal should be to “improve over time” – hence, the next time there is interaction with the customer, a process should kick in to enable the customer details to be verified and corrected.

    Let us assume that there is ongoing population of data in the new system from the legacy system (as in the case of many AML systems). If the above steps are carried out, no more ‘bad data’ will be fed into the new system, and the exceptional processing will no longer execute. Ideally, one would remove it – but there is no real need to.

    I hope I understood your questions correctly, and I hope you find the answers useful.

    Rgds Ken

  6. Hello all.
    Ken, as promised I will chime in more on this topic as time permits over the next couple of weeks. I just wanted to share some recent “wins” with regards to the internal/bespoke rules you mentioned earlier.

    We recently completed a the process of gathering and examining all bespoke rules around our customer data (implemented across >7 systems). With the help of a legal/policy analyst, we traced the rules back up through a plethora of acts, regulations, treaties, jurisprudence etc. In many cases we found that the rules, while believed to be “legal requirements” were not in fact a true representation of the requirement. Many of these beliefs are entrenched in business process that have been around (in some cases) for over 100 years. Systems were built over time without questioning these rules, thus cementing the belief.

    In most cases we’ve managed to replace bespoke rules with pre-defined standards. We’ve been capturing revised rules in a requirements management repository (as enterprise level rules). We trace these down to the systems in which they’re being gradually implemented, up up to to the business process that use the rules.

    I’m looking forward to reading & contributing more to the discussion on this excellent topic.

    Cheers,
    Marianne (@emx5)

  7. Hi all,
    What are the most popular choices of software for handling the recording of Business Rules and what experiences do you have in that regards?
    How useful/popular is OCL in this respect?

    Phil.

    • Hi Phil.
      In my curent environment we use Borland CaliberRM for capturing business rules. We’ve integrated this with Holocentric to trace rules back up to Business Processes.
      I’ve previously used Telelogic DOORS and IBM (Rational) Requisite Pro which also fit the bill quite well.
      Cheers,
      Marianne.

  8. Pingback: Eligibility Business Rule – Gobbledygook! « Ken O'Connor Data Consultant

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s