After a missed call, the customer gets an SMS with the real ship date

The key finding: the order has no ship date

The central question of the whole system is where to get a date you aren't embarrassed to send a customer. It turned out that a Shopify order has no calendar ship date anywhere. I checked four other sources, and none of them held the date the customer had been promised.

The promised date lives in only one place: the name of the shipping option the buyer chose at checkout. That is where they saw the timeframe and agreed to it. The date is worked out from that text. The whole two weeks of work hinged on this one finding, and it had to come before any code was written.

The reply is built in steps

The reply is chosen step by step, from the most specific to neutral. No step is allowed to mislead the customer:

• customer and order found, promised date still ahead → give the date
• promised date already passed → don't repeat it; say honestly that we're checking the date with the warehouse
• the order exists but no date can be worked out → mention the order without inventing a promise
• no customer matched the number → a neutral reply, and the call goes into the queue for a person

The rule "never repeat an overdue date" was checked against an export of the full order history (more than 11,000 orders): not a single old, stuck order gets a date in the future.

Safeguards on sending

An automatic SMS goes out in the company's name, and nobody sees it before it's sent. So the limits matter more than the text itself:

• quiet hours: messages go out only between 11:00 and 21:00 New York time, which is a reasonable hour in every time zone of the continental US; Alaska and Hawaii are handled separately
• a pause for each number: if someone calls again, they don't get a stream of messages
• an overall rate limit on sending
• draft mode: everything is calculated and logged, but nothing goes out; live sending is turned on with a separate switch

The "Missed calls" page in the CRM

The system sends the first reply itself, but a person still has to deal with the calls. There is a separate page for that: who called and when, the number as a tel: link you can call right away, the matched customer, their orders with status ("unfulfilled · 82 days"), a ready reply and a "handled" checkbox.

The page survives a partial failure: one broken line in the queue doesn't bring it down. If the queue can't be read, the page says so: "queue could not be read". Had it said "no calls", the operator would calmly have closed the tab.

I fixed five bugs along the way, and three of them had stopped the system from working at all

• the customer lookup crashed on mismatched function parameters, so every call was treated as "customer not found"
• event parsing looked at the wrong level of the data and silently threw away every missed call
• the caller's number was read from a field the notification doesn't have
• the auto-reply card brought down the whole calls page, the page two staff members start their morning with. The code was written against guessed field names, and the test data was made up the same way, so the test only confirmed the author's guess
• the missed-call totals were written and covered by tests, but nothing called them. The manager kept asking how much the missed calls were costing and got no answer, although it was already in the code

The first three were invisible from the outside: the service ran, logs were written, and no SMS went out. Everyone noticed the fourth at once, because the calls page stopped opening. The fifth showed up only as silence: the code for the totals existed, but nobody ever saw the numbers.

How to check that notifications are getting through

Silence in the system can mean two things: there really were no calls, or the notifications stopped arriving. As long as the two look the same, a report about a quiet night is worth nothing. So the delivery of notifications is checked separately:

• The receiver ran code almost three weeks old for 18 days. The restart counter showed zero, and the file checksum on disk matched the repository, but that proved nothing: you have to compare the process start time with the file modification time. The consequence was exactly what this case is about. The customer lookup by phone number went out as a request to another server, an empty response made the function silently return nothing, and the call turned into "customer not found". Before the restart, the code was checked against the repository byte for byte, and 732 tests of this part of the system were run; all of them passed
• The "were there any calls" check relied on a service method of the phone system's API that returns HTTP 400 with no data. A simple parser prints "0 calls", and the error can't be told apart from proof that no calls came in. This came to light during a control period when calls had definitely come in, yet the method still showed "0". A different method is now used for the check, and notification delivery is verified in four independent ways, none of which depends on the queue
• The trigger rule was written down in advance: "if no event at all has arrived by 18:00 Monday, check the notification settings". That check was never needed. The weekend explanation held exactly: Friday 4 events, Saturday 0, Sunday 3, Monday 6, and the 41 hours of silence fell on the weekend
• The watchdog script waiting for an email from the phone provider read the very message it was waiting for, saved it and alerted nobody. The unsubscribe filter searched for its phrase across the whole email, and the provider's ticket system appends the entire earlier thread below. Now only the new part of the email is parsed. The same check turned up a second problem: the script's log held 50 lines, and every one of them was an error. A successful run wrote nothing, so "ran without incident" and "not running at all" looked the same

The same rule applies as on the queue page: "couldn't read" is never shown as "empty".

When customers actually call

The system had run long enough for the notification log to answer a practical question: which hours should the line be staffed? The measurement uses only the log on disk and calls no outside services. Repeated events are removed by call ID, the last event with a final status counts, and the customer's time zone comes from a phone-number parsing library. That gives 739 inbound calls over 90 days, with the time zone known for every one. Counting everything in one time zone wasn't an option: eight in the morning on the East Coast is five in the morning on the West Coast, and the conclusion would have come out backwards.

• most calls come at 11:00 in the customer's local time; the 09:00–12:00 window carries 38% of calls
• 82% of calls fall in business hours, 5% in the evening and 1% at night in the customer's local time, so the idea that people call in the evening didn't hold up
• Monday gets almost twice as many calls as Friday
• 496 of the 739 calls came in during the advertised business hours, and 310 of those were missed
• 530 calls were missed over the 90 days in total, about 180 a month, the same figure the first phone export showed
• in the daily report, 7 of 9 missed calls fell within the advertised business hours

That gives the number all this was for: if the line closed two hours later, at 17:00, it would have caught 133 more calls over 90 days, about 44 a month. Holidays are calculated by formula, because a table of dates is the kind of thing people forget to extend.

I made a mistake here myself, and fixing it took an extra round. The first version worked out time zones from my own table of area codes and immediately put one of the cities in the wrong zone. A time-zone error quietly shifts the whole peak and leaves no trace on the chart. So I threw the table out and handed number parsing to a library, a Python port of libphonenumber. The report notes one caveat that I couldn't check: an area code may not match where a person actually lives.

The "Yesterday" card on the calls page

Next to the queue there is now a summary of the last 24 hours: how many auto-replies went out, how many didn't, and why. The card collects data from every source, and its test checks exactly that. With all the sources connected, the test expects "not sent: 2" with two reasons. If one source is replaced with a stub, the card shows "not sent: 0", and the test catches the gap at once.