Long-exposure ribbons of blue and white light.

A duplicate contact costs more than a missed one

An enquiry nobody answers is lost once, and the loss stops there. A duplicate record keeps costing after that point, because every sequence built on the CRM treats one person as two, so we agree the merge rules with the client and run them before anything goes live.

Muhammad Hussain, Partner and CEO6 min read

A missed enquiry is lost once

An enquiry that nobody answers costs what it cost to bring in. The ad was paid for, the form was filled in, the message sat in a shared inbox over a weekend, and by Monday the person had bought from whoever replied first. The loss is bounded. Nothing further happens to that person, and the opinion they hold of the company is the one they formed when nobody wrote back.

A duplicate record is a different kind of failure, because the damage carries on past the point where the missed enquiry's damage stopped. One person exists twice in the CRM. Every automation built on it treats them as two people, so they get two acknowledgements a few minutes apart and, later, two chases on the same quote. The person receiving them does not read it as a data problem. They read it as a company that doesn't know who it is talking to.

The missed enquiry was also only one enquiry. A duplicate problem is a share of the whole database, and a sequence reaches all of it on the day it goes live. That is why we settle duplicates first, and why we say so in the strategy session even when the client came to talk about follow-up.

How duplicates get made

Duplicates are rarely carelessness. The same person fills in the web form, emails the shared inbox two months later from a different address, phones the office and sends a message on a social account. Each route creates a record if nothing checks first, because the form integration creates a contact on every submission and the inbox sync creates one for every sender address it hasn't seen before.

Small differences defeat a simple match. A capital letter in an email address, a trailing space, a work address and a personal one, a first name typed in full on one form and shortened on the next. Every one of these looks like a different person to a system that matches on exact text, which most CRMs do unless they are told otherwise.

Imports do the rest. A list exported from an old system, a spreadsheet somebody kept on their own machine for two years, a batch of business cards typed up after a trade show. Each gets pasted into the CRM with the duplicate check switched off because it slowed the import down. We have sat with teams where the largest single source of duplicates was one import nobody remembered doing until we found the created date.

Most of these routes get past the CRM's own check, which runs on one field at the moment a record is created, and the count grows quietly until something automated starts sending.

What two sequences do to one person

The sequence itself is usually built correctly. It stops on a reply, on a booking and on an unsubscribe, and the number of messages is capped. But it stops on the record that replied. The other record keeps going, so a person who answered the first message and booked a call still gets a chase three days later asking whether the quote arrived, from a company they're mid-conversation with.

Unsubscribes are where the failure is worst. An unsubscribe attaches to the record and not to the person, so someone who unsubscribes from one record stays subscribed on the other and the next message goes out on schedule. That is the message that produces a complaint. The company was told to stop, in writing, and carried on.

None of this shows in the dashboard. The report counts two contacts and two sequences started against one reply, so the reply rate reads as half of what it was. The history splits too, with the call notes on one record and the quote on the other. The team takes the low rate as a problem with the wording and rewrites the messages. The wording was fine.

The merge rules that run first

So before a sequence goes live we run a merge, with rules the client agrees in writing, because a merge is destructive and whatever is not kept is gone. The first rule is what counts as the same person. An email address in lower case with the spaces stripped out is the primary match, and a phone number reduced to its digits is the second. Anything looser, a matching name at the same company with a different address for instance, is flagged for a person and never merged on its own.

Which record survives is the second rule. We keep the oldest, because it usually holds the history, and pull the newer fields onto it. Where the two disagree, the newer value wins for a phone number or a job title and the older wins for the source and the created date. Notes, emails, calls and deals from both records go onto the survivor, and nothing is deleted until the merge log has been read.

The third rule covers the sequence state. If either record has unsubscribed, the merged record is unsubscribed. A reply on either side marks it as replied and stops any sequence still running, and a bounce on either address stops anything further going there. An open deal on either record comes across, and the record is kept out of the new-enquiry sequence. The stricter status always wins. A CRM's default merge gets that wrong, because it keeps the survivor's status whatever the other record said.

Merging is not a one-off, and that is the fourth rule. New duplicates arrive every day through the same routes, so the same matching runs every night against the day's new records. Whatever matches is merged by rule or queued for somebody in the client's team to look at, and that somebody is named before the build. All four rules go into the handover note in the fields and filters they run on, so a person who didn't write them can change a threshold without calling us.

What the merge does not fix

The routes stay open. A merge is a repair. The form integration still creates a record on every submission and the next import still arrives with the check switched off. The lasting fix sits upstream, in a lookup before every create and an import template with the check on, and we build that where the tools allow it. Where they don't, the nightly merge carries the load.

Two records with the same name at the same company and different email addresses might be one person with two addresses or two people who share a name. A rule that merges them automatically will eventually merge two different people. That is worse than either failure above, because one person's quote history is then on another person's record and a message about it goes to the wrong one. Those cases go to the queue and a person decides.

A CRM with the duplicates merged still has the empty first-name fields and the contacts nobody has spoken to in three years. The merge comes first because it is the cleaning job a live sequence turns into a complaint fastest, and the rest follows in the order that each sequence going live makes the next problem expensive.

What we do

Topic:

Data and CRM

About the author

Muhammad Hussain is Partner and CEO of Nuha Consulting and an expert in marketing automation. More about Muhammad

“Where two records disagree on whether a person has replied or unsubscribed, the stricter status wins the merge, and that rule is agreed with the client before any sequence goes live.”

Muhammad Hussain

More insights

Read more
The next meetings are in contacts untouched for twelve months Rows of metal index cards with one card lifted and lit in blue. 6MIN READ Data and CRM The next meetings are in contacts untouched for twelve months
A segment is only as good as the field it is built on Rows of metal index cards with one card lifted and lit in blue. 8MIN READ Data and CRM A segment is only as good as the field it is built on
An unsubscribe that still receives a message is a bigger problem than a slow reply Rows of metal index cards with one card lifted and lit in blue. 9MIN READ Data and CRM An unsubscribe that still receives a message is a bigger problem than a slow reply
Public records show a need before anyone advertises it Rows of metal index cards with one card lifted and lit in blue. 7MIN READ Data and CRM Public records show a need before anyone advertises it

Talk through one of these processes

If a marketing team still does any of these by hand, we go through it on a first call and say how much of it could run on its own.

Schedule a 20-minute call (opens in a new tab)