You may have heard of the Data, Information, Knowledge, Wisdom (DIKW) pyramid, but just imagine what kind of wisdom you would — or more likely wouldn’t — gain if the quality of the data at the bottom of the pyramid is poor.
Whether it’s data concerning supporters and their donations, students and their course information, or tenants and their housing needs, in all instances data is key and its quality is crucial.
In this blog we will look at data quality concepts and how the Salesforce platform and ecosystem can help to set up and maintain a data quality process.
First Steps to Set Up a Data Quality Process
Data quality rules are the key to data quality. A data quality rule is simply a statement in plain language that defines some attributes about the data you’re using. For example: “The Marital Status field may have values of Single, Married, Widowed, and Divorced. It may not be left blank. A value must be selected when entering a new customer.”
In some instances you may be able to use industry defined rules for data, for example you could simply state that a phone number must meet and be entered in E.164 format.
However, it’s likely that you will need to define rules specific to your organisation and how you want the data represented in your systems. Also, don’t forget about globalisation if your organisation works across countries — not all addresses follow the same format!
The diagram above shows the steps you take as part of the data quality process. Let’s look at the steps involved, and remember: data is everyone’s responsibility, so get the users involved.
- Profile: We need to know what the quality is like, which means profiling the data — analyse your data to determine what the quality is, and should be, based against the established data quality rules.
- Cleanse: Make sure there are no duplicates or incorrect, bad, missing or irrelevant data. Standardise: Create naming conventions and data standards, enforced with validation rules and picklists, and train all users.
- Match & Merge: Create your golden record, match and merge duplicates (where appropriate), and consider Master Data Management principles.
- Monitor: Establish process and controls to measure the ongoing quality of data
In my experience, if you can get this correct and working well, then you should be able to quickly establish good quality data. Keep in mind that this is an ongoing process, so please ensure you allocate sufficient time.
How Can Salesforce Help?
Let’s have a high level look at the features which the Salesforce platform and our ecosystem can offer to help you establish and maintain your data quality. And remember: if you can’t do what you need to do with out-of-the-box features, then the AppExchange should be your first port of call.
Reporting and Monitoring
Data Quality Analysis Dashboards. This app provides a series of preconfigured dashboards, and by leveraging custom fields on many standard objects you can measure data quality and record completeness. These dashboards help when profiling data to understand the scale of any data quality issues, and also can be used for ongoing monitoring.
Validation Rules
Validation rules can be used to simply ensure a field has the required value in it, particularly if the data in the field is dependent upon other data or statuses in the record. Check out some examples here, and when writing your validation rules. Consider putting an extra check in the rule that allows you to use a custom setting to bypass validation rules in particular circumstances, i.e. for mass data loads of clean data.
Data Cleansing and Advanced Validation
Where validation rules can help you check the format of a post code, or finance details, they can’t confirm that the data is actually valid. For this we head to the AppExchange and our partner products that can perform address, email, bank verification to name but a few.
These solutions tend to work in a similar way, an app installed into your org to do lookups of the relevant data in their cloud based solutions, and most allow you to do bulk verification, which is great for your cleanse phase.
Duplicate Management
Salesforce has duplicate management out of the box, and can provide you with a simple alert on a duplicate record or prevent the duplicate being entered.
Duplicate management is made up of two elements: The matching rule, which defines the object and which fields you want to look for duplicates — this can either be an exact or a fuzzy match. Secondly, duplicate rules use the matching rules and determine what will happen on a match, i.e report, alert and/or allow.
If you need more functionality, then again check out the AppExchange for some of our partner duplicate management products.
Depending upon your business process, duplicate records may be necessary, so ensure you can manage this, and identify the records in your business processes.
Data Usage
Salesforce Optimizer is a useful tool to help you understand completeness, and timeliness. Using the data list section you can see field usage to determine which fields are empty.
Some Final Thoughts
- Choose the correct Field Data types, and avoid free text if possible.
- Don’t forget reference data. Whether in a picklist, multi-select picklist or using an object for reference data, ensure you apply the same level of data quality.
- Don’t forget about ‘non-traditional’ object records, for example email messages, tasks, events or metadata.
- Remember the user experience. Use the out-of-the-box features to help/inform your users, and in some circumstances customers, to ensure they are entering the correct data. Also consider using a flow to ease the process of capturing data.
And remember this quote from Tim Berners-Lee: “Data is a precious thing and will last longer than the systems themselves.”
