What Is Data Matching
Data matching is the process of identifying and consolidating records that refer to the same company or individual from customer information dispersed across multiple databases and systems. In English, this concept is often referred to as Entity Resolution or Record Linkage.
In the B2B sector, the majority of business partners are corporations rather than individuals. Therefore, data matching primarily focuses on corporate matching, which involves grouping multiple records that refer to the same corporation or business location.
The fundamental reason data matching is necessary is that even for the same company, naming conventions can vary depending on the system used or the person entering the data.
Typical discrepancies addressed by data matching include the following:
Variations in Company Name Notation
Countless notation patterns can emerge for the same company, such as whether to place 'Co., Ltd.' at the beginning or end, the mixing of full-width and half-width characters, differences between English and Katakana spellings, and the presence or absence of spaces and symbols.
Even if a human can recognize that these refer to the same company, systems will treat them as separate entities.
Hierarchy of Business Locations and Branches
Within the same corporate group, multiple locations exist, such as headquarters, branch offices, sales offices, and factories. Determining which unit is considered 'identical' varies depending on the purpose of use, making it essential to accurately capture the hierarchical structure.
Trade Name Changes, Mergers, and Acquisitions
Companies change their names and legal entities through rebrandings, mergers, and acquisitions. To correctly link historical data with current data, a system capable of tracking these transitions is required.
Presence of Duplicate Records
When integrating data from business card management tools, exhibition lists, and past transaction histories, it is common to find multiple records registered for the same company. Performing analysis or executing strategies while duplicates remain can lead to results that deviate significantly from reality.
Click Here for Past Articles on Data Cleansing
Practical Process of Data Cleansing
Data cleansing is generally said to be executed through the following process.
Data Collection and Inventory
We identify the location of customer data scattered across your organization, including CRM, SFA, MA, core systems, and business card management tools. Understanding which system holds which data fields is the essential starting point.
Cleansing and Normalization
We perform 'Data Cleansing' to standardize data formats, such as unifying full-width and half-width characters for company names, removing unnecessary symbols and spaces, and organizing corporate entity designations. This preprocessing significantly improves the accuracy of subsequent matching.
Matching and Assignment of Consolidation Keys
Using normalized data, we identify identical companies by combining attributes such as company name, address, and phone number. The foundation for this identification is the 'Consolidation Key (Company Code)' uniquely assigned to each enterprise. Assigning a common key enables data reconciliation across different systems.
Integration and Finalization of Consolidation Results
We group records identified as identical and determine the representative information. By defining rules for integration levels—such as which branch units to consolidate and which data fields to prioritize—we create consistent integration results.
Continuous Updates and Monitoring
Data consolidation is not a one-time task. By reflecting changes in corporate information and continuously assigning consolidation keys to newly added data, you can maintain high data quality.
Use Cases for Data Consolidation by Industry
The requirements for data cleansing vary by industry.
Here, we outline five representative industries and the common scenarios where data cleansing is utilized in the field.
Manufacturing
In the manufacturing industry, indirect sales through trading companies or distributors often coexist with direct transactions with end users, creating a structure where it is difficult to identify who is actually using the company's products.
By cleansing transaction data separated by sales channel, you can visualize the customer profile, including the end users beyond the distributors. Additionally, since headquarters, factories, research laboratories, and various business sites are often registered as separate business partners, consolidating transactions at the corporate group level allows you to grasp the transaction scale and cross-selling opportunities across the entire group. Data cleansing also plays a crucial role in correctly linking after-sales service and maintenance history by product and location.
Financial Services
In the financial services industry, accurately identifying corporate clients is essential for identity verification, anti-social force checks, and credit assessment as required by law. When systems are siloed by product—such as loans, deposits, payments, and insurance—the same client may be managed as separate customers, leading to the challenge of being unable to grasp the risk (credit concentration) of the entire corporate group. By consolidating group companies and affiliates through data cleansing, you can improve the accuracy of exposure management at the group level and gain a better understanding of the actual status of your business partners.
Furthermore, identifying customers uniquely across multiple accounts and contracts is considered essential for proposing appropriate financial products and detecting fraudulent transactions.
Construction
In the construction industry, numerous companies are involved in a single project, including prime contractors, subcontractors, partner companies, and material manufacturers. It is common for the same partner company to be registered with different spellings for each site or project, causing transaction records to become fragmented. By correctly integrating partner companies and business partners through data cleansing, you can centrally manage order history and safety management records at the corporate level. Given the industry-specific complexities, such as information on licensed construction contractors and the handling of business partners formed as Joint Ventures (JV), there is significant value in identifying entities by cross-referencing them with a reliable corporate master database. Data cleansing is also utilized for consolidating suppliers in material procurement and for credit management purposes.
Human Resources Services
In the human resources industry, which handles recruitment, staffing, and job advertising, companies manage both corporate clients and job seekers. On the client side, the same company is often registered as separate accounts by different departments or recruiters, making it difficult to grasp the overall business relationship. By using data cleansing to consolidate transactions by company rather than by department, you can avoid redundant proposals and build relationships across the entire account. Furthermore, by cleansing and normalizing the names of employers listed in job seekers' resumes, it is possible to improve the accuracy of resume matching and perform screening based on attributes such as industry and company size.
Government and Municipalities
In government agencies, municipalities, and public financial institutions, business entity information is often scattered across multiple operational systems, such as those for subsidies, licensing, bidding, and taxation. When the same business entity is managed differently across various systems, it hinders a comprehensive understanding of the actual situation and the streamlining of procedures. By uniquely identifying business entities through data cleansing, you can consolidate various applications, prevent the duplication of benefits, and establish a foundation for analyzing business data for policy formulation. In recent years, the development of the Corporate Number system has laid the groundwork for uniquely identifying businesses. However, it is said that a data cleansing mechanism capable of handling variations in notation and historical changes remains essential for integrating past data and information on businesses that do not possess a Corporate Number.
Conclusion
Data cleansing is a foundational process that supports the accuracy of data utilization by correctly grouping scattered customer data into units of the same company or the same individual. To overcome challenges such as variations in notation, data degradation, and the absence of a master record for judgment, a mechanism for continuous data cleansing based on reliable corporate information is indispensable. Furthermore, as seen in this article, because the challenges addressed by data cleansing manifest differently across industries, it is considered crucial to design its application in accordance with your company's specific business structure.
Our customer data integration solution, uSonar, can address these data cleansing challenges with high precision. Through our proprietary code system, the Linkage Business Code (LBC), we uniquely manage corporate information at the business establishment level. This allows for data cleansing across systems by absorbing complex patterns such as variations in notation, branch hierarchies, and changes in trade names.
Because data cleansing is performed based on one of Japan's largest corporate information databases, it is possible to identify companies with the same name or different names for the same company with high accuracy—tasks that are difficult to distinguish using only your own internal data. Furthermore, by continuously acquiring our independently collected corporate data, you can keep up with changes such as relocations and name changes, maintaining your data cleansing results with high freshness.
For more details, please feel free to contact us via this page.
