What constitutes a duplicate, and what doesn't?

This is a business decision, not merely technical. Two branches of the same corporation might be legitimate separate records for sales, but a single record for billing. Before running any deduplication tool, you must define the core entity: Is your business unit the corporation, a specific site, or a contractual agreement?

Should we cleanse data before or after loading into Salesforce?

Always cleanse BEFORE. A duplicate record loaded into Salesforce already creates relationships – activities, cases, opportunities. Any late-stage merge becomes a risky operation with potential history loss. Post-load, your focus should shift to ongoing enforcement, not large-scale cleanup.

What if two records hold different, yet equally valid, information?

Build a 'Golden Record' at the field level, not just the record level. For each field, define a precedence rule: preferred source system, most recent update, or a validated value. This ensures you don't lose a correct address simply because the 'winning' record had an outdated one.

Are Salesforce's native Matching Rules sufficient?

Salesforce's native matching rules are excellent for ongoing prevention during manual data entry, but they're typically insufficient for historical data cleanup before a major migration. Initial cleanup requires a dedicated tool or script capable of fuzzy matching on normalized text, allowing for human review of 'gray area' records before merging.

What's a 'normal' percentage of duplicates?

In older, unmanaged databases, it's common to see 8-20% duplication at the Contact level and 3-10% at the Account level. The practical goal isn't zero, but an agreed-upon threshold – for example, below 2% – supported by a process to prevent regression.

Pre-Salesforce Deduplication Strategy: Data Cleansing

The Short Answer

Deduplication fails when treated as a one-time cleanup. It's actually three decisions: what defines identity, who wins when there's a conflict, and how to prevent the problem from recurring. The technical tool is the easy part.

The common mistake is to run Fuzzy Matching on names, get a list of 12,000 potential matches, and try to manually resolve them under time pressure. What works is the opposite: first narrow down the decision space using strong keys, and leave only the gray area for human review.

The broader migration plan is described in Salesforce Data Migration Guide.

Three Layers of Matching

Layer	What it relies on	Action
Strong Key	Company ID, VAT ID, source system ID, verified email	Automatic Merge
Complex Key	Normalized Name + City + Normalized Phone	High-score Automatic Merge
Textual Similarity	Name only, free-text address	Human Review Only

A well-managed project typically looks like this: approximately 70% of duplicates are resolved in the first layer, about 20% in the second, and 10% reach a human. If most matches end up in the third layer, it's a sign that normalization wasn't thoroughly implemented—not necessarily that the data is exceptionally poor.

Normalization Before Comparison

Before any comparison, create normalized helper columns without altering the original data: remove corporate suffixes (Ltd., Inc.), standardize spaces and apostrophes, format phone numbers to E.164, convert email to lowercase with plus tag removal, and split addresses into street/number/city. For Hebrew, add handling for full/missing vowels and acronyms.

Normalization alone typically reduces one-third to one-half of "hard" duplicates even before applying any similarity algorithm.

Golden Record at the Field Level

The decision of "which record survives" isn't the critical one. The critical decision is "which value survives in each field." Establish a concise policy: billing details from the ERP, contact information from the system where the last activity was recorded, customer status from the operational system. Each field should have one preferred source, and any rejected values should be documented.

Without such a policy, every merge becomes a decision made by the person performing it at that moment—and it's impossible to explain later why an address disappeared.

What Happens to Relationships and History

Merging records affects Activities, Opportunities, Cases, Files, and Permissions. Before a bulk run, explicitly define: where activities go, what happens to open opportunities for the same customer from two records, and who the owner is after the merge—because changing the Owner also impacts visibility and commission reports.

The practical rule: no merging before a reproducible "what changed" report exists, and source identifiers are preserved in a separate field to allow for investigation months later.

Scenario: Importer with 210,000 Contacts

A B2B importer began a migration with 210,000 Contacts from three systems. The first run of a similarity tool returned 31,000 suspicious pairs—a number no one could review.

The team stopped and reversed the order. First, email and phone were normalized: 14,000 pairs were automatically resolved by a strong key. Then, it was determined that the business unit was a customer site, not a corporation, which removed 6,000 legitimate pairs from the list—separate branches of the same chain. This left 4,200 pairs for the intermediate layer, of which 3,800 were resolved with a high score. 400 pairs reached human review, and two people resolved them in three days.

The lesson wasn't about tool selection. It was that defining the business unit—site versus corporation—reduced more noise than any algorithmic improvement.

Prevention: Why Duplicates Return

Three main sources cause duplicates to return after Go Live: manual entry without active Matching Rules, integrations that create new records instead of updating (Upsert on External ID solves most of these), and Web-to-Lead forms without existence checks. If these three aren't addressed, the duplicate rate will return to its original level within one to two years.

Common Risks and Prevention Actions

Risk	How it appears in practice	Prevention Action
Aggressive Merge	Different customers merged and cannot be separated	High score threshold + gray area review
No Identity Definition	Recurring debate over what constitutes the same customer	Documented entity-level decision
History Loss	Activities and opportunities disappeared during merge	"What changed" report and preservation of source IDs
Cleanup without Prevention	Duplicates return within months	Matching Rules, Upsert, and protected forms
Cleanup After Load	Every merge touches live relationships	Clean in the Staging phase

How to Measure Success

Area	What to Measure	Check Frequency
Uniqueness	Estimated duplicate rate per entity	Weekly during migration, quarterly afterwards
Merge Accuracy	Percentage of merges undone or manually corrected	With each merge wave
Prevention	New records blocked as duplicates during entry	Monthly
Business Impact	Duplicate customer outreach, accuracy of customer reports	Quarterly

Professional guidance in building identity and prevention rules is available through our Integrations and Data Services.

Checklist Before Running a Merge

☐ Business unit defined: company, site, or contract
☐ Normalization columns built without altering source
☐ Three layers of Matching with written numerical thresholds
☐ Field-level Golden Record policy, business-approved
☐ Decision made on what happens to activities, opportunities, and ownership
☐ Source IDs preserved after merge
☐ "What changed" report can be generated and reverted
☐ Trial run on a sample with manual verification
☐ Matching Rules and Upsert active for prevention
☐ Permanent owner for the process after Go Live

Professional Resources

Salesforce Data 360 — https://www.salesforce.com/data/
Salesforce Data 360 Architecture — https://architect.salesforce.com/docs/architect/fundamentals/guide/data-360-architecture.html
HPI Pro – Integrations and Data — https://hpi.pro/integrations-data

Pre-Salesforce Deduplication Strategy: A Practical Guide to Data Cleansing