CRM Data Hygiene: Duplicates, Account Matching and the Merge You Cannot Undo

Search your CRM for your largest customer. If three records come back, one with the GmbH, one without, and one with a typo a rep made in 2023 while on a call, you know the state of the database. The interesting question is not how to clean it, but why it refilled within two quarters of the last clean-up.

Lead routing names deduplication in a paragraph. This is the whole problem.

Duplicates are produced by the system working as designed

Almost none come from carelessness. Five mechanisms account for nearly all.

Form fills with a personal address. A buyer downloads something on the train from a private address, then returns two months later with a work address. Two people, to any matching rule.

Imports. A trade fair list, an agency's spreadsheet, a leaving colleague's contacts. Import tools match on exact email by default, so every different address becomes a record and every different spelling an account.

Enrichment. A provider returns a canonical company name and a corporate email that differ from what you hold, and the integration writes a record rather than updating one. The most avoidable source, and the commonest.

Sales creating what already exists. A rep searches for "Müller Logistics", finds nothing because the record says "Mueller Logistik GmbH", and creates it. The search failed, not the person.

Mergers and renames. A customer acquires someone, rebrands or changes legal form, and you hold two accurate records of one relationship, neither wrong.

Person matching is nearly solved, account matching is not

At the person level it is tractable: normalise the email, lowercase it, strip plus-addressing and match exactly, with the name as a secondary check. Accounts are hard, because no equivalent key exists.

Domains are not companies. A group uses one domain across fifteen subsidiaries you sell to separately. A local subsidiary uses a country domain while the parent uses the .com. A reseller emails from its own domain about your product, and freemail and agency domains belong to nobody.

Legal names are unstable. GmbH, AG, SE, Ltd, B.V., S.à r.l.: suffixes appear, disappear and change with legal form. Umlauts get transliterated, punctuation varies. "Schmidt & Söhne" and "Schmidt und Soehne" are one buyer sharing almost no characters a naive comparison respects.

Your commercial unit may not be a legal one. If you sell per site, per country or per business unit, the correct account is a decision, not a fact, and teams that never make it explicitly argue for years about duplicates that are not duplicates.

Deterministic is safe, fuzzy is fast, and a false merge is worse than both

Deterministic matching compares normalised values for equality: an email, a domain, a registration number. It is right or it finds nothing, which makes it safe to automate.

Fuzzy matching scores similarity across name, domain and location, and catches what deterministic rules miss. It also produces the failure nobody plans for. A false merge fuses two real companies into one record and, unlike a duplicate, is close to impossible to unpick: the histories are now interleaved, and the CRM's undo, where it exists, restores the record but not the relationships that hung off it.

Let the asymmetry drive the design. A duplicate costs a confused rep and a slightly wrong count; a false merge costs an opportunity history and a customer receiving a sequence written for another company. Automate deterministic matches, route high-scoring fuzzy ones to a human queue, and let the rest sit. Ten unmerged duplicates are cheaper than one bad merge.

Fix it at entry, because a quarterly clean-up is a treadmill

Every clean-up ends the same way: a good month, then steady refill, because the mechanisms producing duplicates were never touched. The work belongs at entry.

Match before create, on every path. Forms, imports, enrichment writebacks, API calls. A real-time lookup on normalised email and domain at submission costs milliseconds and prevents most of it.

Make account search work. It should strip legal suffixes, handle umlauts and match on domain as well as name.

Stage enrichment and quarantine imports. Provider output goes to staging fields and is promoted on a rule, so you can tell what a human typed from what a vendor asserted. No import lands directly in production objects either: it is matched in staging, and a person approves the residue. That is what a signal ledger does for events.

A free-text country field is a reporting bug with a delay

Required fields and picklists are the schema your reports depend on, and country is the standard casualty. Leave it free text and within a year you hold Deutschland, Germany, DE, GER, "Germany ", and a blank. Nothing breaks visibly, and then territory routing sends records to a default owner, the regional report undercounts DACH, and someone spends a day on a normalisation query stale by next month. Use ISO codes from a picklist, validated on entry.

The same applies to industry, size band and lead source. If a value feeds routing, scoring or a report anyone reads, it is a controlled vocabulary. Free text is for context a human reads, never for a field a system groups by. Require only what routing needs, though, since each mandatory field costs completion, and derive the rest from enrichment.

Survivorship rules, decided per field before you need them

A merge is not one decision but one per field, and CRMs default to keeping the primary record's value, which is usually wrong.

Write them down. Original source and created date from the oldest record, the field every attribution report reads and the one most quietly corrupted. Contact details from the most recent non-null value. Owner from whichever record has an open opportunity. Lifecycle stage from the furthest-advanced record, never the newest. Custom fields listed explicitly, since unlisted ones inherit a default nobody chose.

Then learn what your CRM silently drops. Activity history, notes, attachments, campaign memberships and custom object links are handled differently in every system, and some are not carried at all. Test a merge in a sandbox on a fully populated record and diff the result before the rule reaches production. Snapshot the loser, so a bad merge is recoverable in principle.

The ten-minute check

Export account names and domains to a spreadsheet. Strip legal suffixes, punctuation and spaces, lowercase the result, and count groups with more than one row. That finds the GmbH-versus-no-GmbH pairs and the trailing-whitespace twins.

Then group by email domain and count distinct account names per domain: anything above one is a duplicate or an unmodelled subsidiary.

The number is normally five to fifteen per cent of accounts. Above that, the entry points are unguarded, which the readiness audit confirms in detail.

What no tool fixes

Deduplication software finds and merges. It does not decide what an account is, and it cannot survive a process that rewards creating records.

If reps are measured on accounts added, duplicates will be created, because no matching rule outruns an incentive. If marketing is measured on database size, nobody proposes deleting.

The durable fix is unglamorous: one named owner of the account object, a written definition of an account, matching at every entry point, and a merge policy tested before it is needed. The software is the last ten per cent, and buying it first produces an efficient duplicate generator.

Frequently asked questions

Why does my CRM keep filling up with duplicates?

Because clean-up projects never touch the mechanisms creating them: form fills from personal addresses, imports matching only on exact email, enrichment writing new records instead of updating them, reps creating accounts their search failed to find, and customers who rebrand. Until matching happens at every entry point, the database refills within two quarters.

How do you match company records reliably?

Start deterministically on values meaning the same thing everywhere: normalised email domain, VAT or registration number. Normalise names before comparison by stripping legal suffixes, punctuation and case and transliterating umlauts. Treat fuzzy scores as suggestions for a human queue, not automatic merges. And decide first whether your account is the legal entity, the group or the site, since matching cannot be designed before that.

Is a false merge worse than a duplicate?

Yes, by a wide margin. A duplicate causes a confused rep and a slightly inflated count. A false merge fuses two real companies permanently: histories interleave, opportunities attach to the wrong buyer, and CRM undo restores the record without the relationships hanging off it. Automate deterministic matches only, queue fuzzy ones, and snapshot the loser first.

What survivorship rules should we use when merging?

Decide per field rather than accepting the CRM default of keeping the primary record's value: original source and created date from the oldest record, contact details from the most recent non-null value, owner from the record with an open opportunity, lifecycle stage from the furthest-advanced one. List custom fields explicitly, and test a merge in a sandbox to see which notes and links vanish.