A single drop rippling across still dark water in blue light.

Two people handling the same reply will not automate it the same way

When the person who does a task says it depends, the dependency is either a rule nobody has written down or a judgment that should stay with a person, and from across a table the two look identical. We settle which it is by giving ten real cases to two people separately and reading their replies side by side.

Muhammad Hussain, Partner and CEO8 min read

It depends is two different answers

Somewhere in the first hour of a strategy session, the person describing how they handle an enquiry or a quote stops partway through a step and says that it depends. We hear it on almost every process a team nominates, and it is the most useful thing anybody says all day, because that word is where the decision about automation gets made. Everything before it in the description was a sequence anyone could follow. Everything after it is the part the team has been carrying in one or two heads.

It depends means one of two things, and the person saying it usually cannot tell us which. Sometimes the dependency is a rule nobody has written down: a same-day reply if the enquiry mentions a date, the longer quote template on any job over a certain size, the account manager copied when the customer has an open invoice, and no price in a reply to anything that came through a partner's form. The person applies all of that without noticing, so when we ask what they do, they say it depends, because from where they sit the rule feels like taste.

Other times the dependency is a judgment. Two enquiries arrive with the same words in the same form and one of them gets a call from the owner, because something in the phrasing sounded like a complaint waiting to happen. There is no rule underneath that, and there won't be one after an afternoon of trying to write it. Automating it produces a sequence that sends the standard reply to the one customer who most needed a person.

The person doing the work cannot separate the two, because from the inside both feel like experience, and we can't separate them from across a table either. Asking harder produces a longer description of the same it depends. So the session stops asking and sets up a test instead.

Ten real cases handled twice

We take ten cases from the last month of the process in question. They are pulled from the inbox or the CRM in date order and nobody picks them for being tidy, because a set of cases the manager chose is a set the manager already knows the answer to. If the process is enquiry handling, the ten are the last ten enquiries. For quoting, they are the last ten requests for a price, with whatever attachments and previous history arrived with them.

Two people who handle that work in the ordinary run of the week then take the same ten cases away and deal with each one as they would on a normal day. Each writes the reply they would send, or notes that they would phone instead and what they would say on the call. They work separately, on the same afternoon if that can be arranged, and neither sees the other's answers until both sets are finished. The one addition we ask for is a line under each case saying what they looked at before deciding, because that line is where most of what we learn ends up.

Where the team has one person doing the job, we use the next closest: the owner, or whoever covers it during holidays. That version of the test is weaker and we say so, because the cover person will handle the cases the way they think the regular person would, and what we end up measuring is their guess. Something still comes out of it. The comparison shows how much of the process exists in one head only, which is what an owner most needs to see before that person hands in their notice.

Reading the two sets side by side

We lay the twenty answers out in a sheet, one row per case, with the two replies in adjacent columns and each person's note on what they looked at beside their reply. Then we go down the rows with both people in the room. The comparison is done out loud, and the question on each row is the same: whether the two of them did the same thing, and if they did not, what each one saw that the other missed.

Cases where both replies match, down to the template and the timing, are rules. The rule was never written, and yet two people applied it independently, which means it exists somewhere outside their taste. We write it down on the spot in the words of whichever of them says it more plainly, and that line goes straight into the build. A step that matches on all ten is the easiest kind to hand over, because the test has already shown the automation will do what two people would have done anyway.

Where the replies differ, the cases split again once the notes are read. Most of the time one person looked at something the other did not. One checked whether the customer already had a record in the CRM and the other didn't think to, so the first sent the returning-customer reply and the second sent the standard one. That is a rule as well, held by one person and unknown to the other, and it is usually the most useful thing the test turns up. The build carries the check the careful person makes, and from then on the check gets made on every case, whichever desk it would have landed on.

What is left after that are the cases where both people looked at the same things and still did something different, and each can say why. One phoned because the enquiry mentioned a deadline and the tone read as anxious. The other sent the quote with a line offering a call, for the same reason. Neither is wrong, and neither can be turned into a rule the other would accept. The difference between them is what an owner pays an experienced person for, and those cases are judgment, so the step stays with a person.

How the count sets the build

By the end of the afternoon each of the ten cases has been called either a rule or a judgment, and the count decides the build. Where all ten matched, the step gets automated in full. Eight matches and two judgments mean the rule gets automated and the exceptions go to a person, and because those two are sitting in the sheet, the routing rule is written from them and not from a guess about what an exception might look like. A split closer to half means there isn't a step yet. It is two people running two different processes under one name, and the work is to agree one before anybody builds anything.

Where the judgment cases are real, the automation is built around them and the person gets everything except the decision. It opens the record, attaches the history, pulls up the last few replies to that customer and leaves a draft in the reply box. What the person does is read the draft and decide. That is the same shape we use for any judgment step, and the ten cases are what tell us where the edge of the draft should sit, because we have watched two people make the decision and can see what each reached for before making it.

The test also changes what goes in the handover note. A rule that came out of the comparison gets written next to the case that revealed it, so that whoever looks after the automation later can see why the check exists and which enquiry it was for. A rule with no example beside it is the one that gets deleted in a tidy-up a year later, by somebody who could not see what it was protecting.

What the test does not tell us

It does not tell us whether the rule is a good one. Two people matching on all ten cases proves the rule exists and is shared, and says nothing about whether the customer would prefer a different reply. We have seen a rule both people applied without fail that was quietly losing quotes, and the test passed it. What to automate is what the comparison settles. Whether the thing being automated is worth keeping is the earlier question, and it belongs to the strategy session.

Nor does it stretch to a process with a hundred branches. Ten cases cover the quoting at a company that sells four things. They do not cover a support desk with a dozen product lines, where the last ten cases from the inbox will miss half the branches and the test will call a judgment what was a rule for a case it never saw. On a process like that we run it one branch at a time, which takes longer and is the honest version.

And it is not an afternoon two people always enjoy. Putting a person's replies next to a colleague's and going down the rows in front of an owner can feel like an appraisal, and there is a version of it where the more experienced person wins every row and the other one stops volunteering anything. We say at the start what the test is for, which is finding the rules that live in one head, and we write each rule down in the words of whoever stated it, including the person who missed it the first time round. A test that makes a junior go quiet has cost more than it found, because the rules that person was holding stay unwritten.

What we do

Topic:

Judgment

About the author

Muhammad Hussain is Partner and CEO of Nuha Consulting and an expert in marketing automation. More about Muhammad

“Where two people looked at the same things and still did something different, and each can say why, the step is judgment and it stays with a person.”

Muhammad Hussain

More insights

Read more
The line between a rule and a judgment call moves as a process repeats A dark stone staircase rising toward a slit of blue light. 7MIN READ Judgment The line between a rule and a judgment call moves as a process repeats
Weekly campaign reports without the manual work A curved row of glass dominoes with the first one tipping. 7MIN READ Marketing operations Weekly campaign reports without the manual work
Which repeating work is worth automating, and which is not A dark stone staircase rising toward a slit of blue light. 5MIN READ Strategy Which repeating work is worth automating, and which is not
The numbers to agree before anything is built Rows of metal index cards with one card lifted and lit in blue. 5MIN READ Measurement The numbers to agree before anything is built

Talk through one of these processes

If a marketing team still does any of these by hand, we go through it on a first call and say how much of it could run on its own.

Schedule a 20-minute call (opens in a new tab)