A 5-Level Guide to Data Cleansing: How to Eliminate Duplicate Data in SFA and CRM Systems

What Is Data Matching?

Data matching (nayose) originated as a financial term referring to the consolidation of multiple accounts held at a single financial institution into one. Today, the term has evolved to also mean the integration of identical data existing within a database, beyond just financial accounts.

Data Matching = Integrating identical data existing within a database into a single record.

Reference Article:
[Explained by a Database Company] What Is Data Matching? A Thorough Guide to Organizing and Managing Customer Data! ▶︎

The Importance of Corporate Data Matching in Database Maintenance

While there are various types of data matching, in the context of "database matching within a company," it can be broadly classified into two categories: individual matching and corporate matching.

Individual Matching

Individual matching involves consolidating records of the same person within a database into a single entry. Typical examples include data at the individual level, such as contact persons or leads. Since SFA and MA systems are fundamentally designed to manage data by individual, individual matching is essential for accurately managing sales contact data.

Corporate Matching

Corporate matching involves consolidating records of the same company within a database into a single entry. Typical examples include data at the corporate level, such as business partners or suppliers. This is crucial for Customer Data Management, preventing sales collisions, and Implementing ABM.

Individual matching is relatively easier to perform because it is easier to set matching keys that are less prone to notation variations, such as Name + Email Address (or Phone Number) (though it is by no means simple).
On the other hand, corporate matching is more challenging because company names and addresses are prone to notation variations, and it becomes harder to set unified keys for small and medium-sized enterprises, government agencies, or specific business locations.

This article focuses on "Corporate Data Matching," which is both more difficult and of higher importance.

Level 0: Define the Purpose of Data Matching

As mentioned above, there are various types of data matching. Since the target data and matching rules change depending on the purpose, it is most important to clearly define "why you are performing data matching" and ensure a common understanding among team members before you begin.

Think of the purpose by working backward from the benefits. Typical benefits include:
● Preventing sales collisions
● Improving data accuracy to enable appropriate marketing and sales activities
● Identifying appropriate new business opportunities through the analysis of existing customers
(You may also need to consider methods for data enrichment in addition to data matching.)

Level 1: Determine the Target Data and Rules for Matching

Determine the Target Data

Decide which data to target for matching based on your objectives. If your internal data is scattered across CRM, SFA, MA, and business card management systems, consider using a data warehouse (a central repository for data).

Determine Matching Keys and Logic

Once you have determined the target data, establish the rules for matching. There are two main components to define: "Matching Keys" and "Matching Logic."

Key: Information used as the condition for determining duplicates.

Logic: Rules for handling scenarios that cannot be covered by keys alone, such as the priority of data when matching keys are identical.

Ideally, matching keys should be set using symbols or codes that are unique and tied to the actual data, such as email addresses for individuals or corporate registration numbers for companies.

Define logic for scenarios that cannot be covered by keys, such as when a matching key is blank or when identical data exists. For example, if using an email address as a key, you might set a rule to prioritize data where the name is not blank when email addresses are identical.

Also, since it is difficult to completely eliminate all data duplication, it is important to define the boundary of how far to perform matching and where to leave the data as is.

Level 2: Perform Data Matching Using Standard CRM or SFA Features

This section covers the actual steps for the matching process. If your company uses sales support systems like CRM or SFA, many tools come with built-in matching features that allow you to manage duplicate data.

Salesforce (Sales Cloud) Matching Features

Here, we will use Salesforce's Sales Cloud, a representative CRM/SFA tool, as an example. Salesforce includes "Matching Rules" and "Duplicate Rules" as standard features for duplicate management.

Matching Rules

Matching rules compare field values to determine if a record is similar enough to an existing record to be considered a duplicate.
For example, a matching rule can specify that records are duplicates if the email and phone values of two records match exactly.

– Sales Cloud: What are Matching Rules?

Duplicate Rules

Duplicate rules work in conjunction with matching rules to prevent users from creating duplicate records.
While matching rules determine if a record being created or updated is similar enough to be considered a duplicate, duplicate rules instruct Salesforce on what action to take when a duplicate is identified.
For example, duplicate rules can block users from saving a record identified as a potential duplicate, or simply alert the user that the record might be a duplicate while still allowing them to save it.

– Sales Cloud: What are Duplicate Rules?

Matching rules and duplicate rules can be configured in Salesforce under [Setup] > [Data] > [Duplicate Management].
You can set field configurations for each object and define exact or fuzzy matching, allowing you to detect duplicates and perform data matching when creating or editing records based on your specific needs.

For more details, please check the Salesforce Guidelines.

Level 3: Export Data and Perform Manual Matching Using Excel

If the standard features of your CRM or SFA have too many limitations to perform satisfactory matching, you can export the data from the system and perform manual matching using spreadsheet software like Excel.

The advantage of spreadsheet software like Excel lies in its flexibility. By utilizing appropriate functions, you can perform more complex condition settings and processing. Representative functions include the following:

Representative Excel Functions

Functions to Standardize Data Notation

Functions to Integrate Data

Manage Items Separately

It is important to manage items that are prone to multiple interpretations separately. For example, having only a "Company Name" field often leads to issues such as inconsistent inclusion of legal entity types or branch names. By pre-defining fields like "Legal Entity Type," "Company Name," and "Branch Name," you can reduce data entry errors. Another major benefit of separating items is that it becomes easier to set data validation rules.

Importing Matched Data

Once you have finished matching the extracted data, import it back into the system. At this time, be sure to import using the unique ID stored in the system. If you do not use the ID, much of the imported data may be generated as new records, potentially creating a large volume of duplicate data.

If you have created new fields, do not forget to create the corresponding fields in your CRM or SFA in advance.

Level 4: Perform Data Matching Using Specialized Tools

Corporate Information Is Updated Daily

There are many types and causes of notation variations and data deficiencies. Corporate information, in particular, is updated daily due to company name changes, relocations, mergers, and bankruptcies.

It is not realistic to capture all these changes and maintain the data using only internal resources.

Using Data Matching Tools...

By using a specialized data matching tool, you can capture changes in corporate information and have the data automatically maintained. By routing data entry through a specialized tool, you can also keep your data clean.
Furthermore, by utilizing the unique codes assigned by specialized tools, you can centrally manage information that was previously scattered across different internal systems, departments, and business locations.

Missing information can be supplemented from the built-in corporate database, enabling more efficient sales activities using data.

For more details on how this differs from manual matching and how to specifically implement it, please check the information below!

In this article, we explained how to perform data matching for internal data in five steps.

Data matching is a deep and complex task the further you delve into it. If you feel that your internal resources have reached their limits, why not consider introducing a specialized tool?