AI Automations Worth Building: Most Do Not Need AI

A public company register gives back raw fields rather than a useful sales list, so every weekday a scheduled worker reads its saved position, asks for the next 500 identifiers, waits a second, and turns what exists into company records. It maps official activity codes into categories, assigns an explainable priority score, and stops before probing empty space all day.

The score is deliberately boring: a valid phone is worth 26 points, a mapped activity category 24, and registration in the last seven days 22. Inactive companies cannot score above 35; companies without a valid phone cannot score above 55, while strong starts at 80 and good at 60. Those are visible business rules that anyone can argue with.

Most valuable automation is not a chatbot on a messy process: it has a repeatable input, stored state, defined transformation and an actionable result, and although AI can belong inside that system, it is usually its smallest and least trusted part. Before paying for an AI automation, ask whether “AI” is doing any work in the sentence.

The valuable part is usually the part people call boring

When a founder wants an AI agent to find leads, update a CRM, chase invoices, prepare reports or route requests, I ask what happens now, who notices failures, and whether the answer needs judgement or merely a rule nobody has written down.

The registration feed sounds intelligent but contains four jobs: discover records without missing a late registration in an earlier empty range; clean and store official data; translate a CAEN code into a human category; and order records by priority. None needs a model.

The scanner persists the highest identifier that produced a company, not the last one probed, because storing the probe position would lose a later registration in an empty stretch. A cursor and careful update rule solve that, while the category mapping is a shared lookup table, the score a function, the insert idempotent, and database triggers maintain the search index.

Judge an automation by whether you can name its input, state, rule and where a bad result becomes visible, because if those are vague, AI hides the vagueness. A scheduled job has a deadline, bounded task and run record, so it can process a small hourly batch when nobody is watching; a chat interface cannot replace that.

A scheduled job and a scoring rule beat a model more often than people admit

Some work is classification by policy; some is classification by meaning. “Is this company active?” is a registry field, a valid phone is validation, newly registered is date arithmetic, and outreach priority can be a transparent score from those facts. A model can imitate that score, not make it accountable.

The deterministic version is cheap, stable across provider model changes, testable at score boundaries, indexable after materialisation, and explainable when someone asks why a company ranked highly. This registration product holds roughly four million rows, so ranking must be an indexed query rather than an API call whose cost and response time vary with traffic; its time-sensitive score refreshes daily only for registrations in the latest 90 days, avoiding unnecessary writes and full-text re-indexing.

The same applies in a smaller business: route support with customer plan, account status, form product area, invoice state, language and open incidents, and that may resolve 70 percent of the workload without reading a sentence. Remind customers about overdue documents with dates, status and ownership, allocate enquiries with territory, capacity and a round-robin pointer, and keep a catalogue current with a webhook and idempotency key.

If the desired output is a spreadsheet formula, I am suspicious of a model, because formulas can be ugly but express a policy an operator can inspect, change and test. A deterministic workflow also creates evidence for a later AI decision: requests that do not fit categories, records that score badly despite becoming customers, and an exception queue that is filling up.

Webhooks need a reconciliation sweep behind them

A webhook is usually the cleanest live data flow: a source sends an address change and your application updates its local record, which is quick, cheap and close to real time, but incomplete.

Webhooks retry, arrive twice or out of order, lose expired credentials, hit a deployment error, or are never sent after the upstream accepts a change, so treating one as infallible is how an integration stays quietly wrong for three weeks.

On a property-inventory operations platform, I replaced legacy integration work with signed webhook endpoints, durable event records, queues, replay controls and dead-letter handling, with reconciliation as its essential companion: a recurring sweep pages upstream operational records, compares them with local state and repairs gaps without becoming a second source of truth.

The sweep respects provider pacing, retains a cursor, marks an expired pagination iterator partial rather than successful, skips unchanged records, and can run again because writes are idempotent, which means a duplicate event or daily rediscovery leaves one canonical record rather than two invoices, two emails or a confused operations team.

The dead-letter queue holds events that have failed enough times that automatic retries are no longer responsible, giving an operator the original payload, error reason and controlled replay path.

I would take this over an “AI operations agent” for nearly every back-office integration:

1. A signed webhook records a change before heavy work.
2. A queue processes it with retries that respect the upstream service.
3. An idempotent projection updates the local record.
4. A dead-letter list holds failures needing attention.
5. A scheduled reconciliation sweep catches events that never arrived or were handled wrongly.

The proof matters most when data is expensive to get wrong: during the property-platform migration, an early image-download path omitted cover images and meter photographs without UUIDs, so the import looked healthy from a dashboard but was incomplete. A per-property manifest and read-only verification found the gap, and remediation was re-verified against 17,595 distinct inspection cover-image URLs and 42,494 meter-photo URLs, while source deletion remained blocked because a separate actions backfill was incomplete.

AI earns its place when meaning is the missing input

Models help when input meaning cannot become a stable field or known rule without wasting a person’s time: an email saying “the report from last Thursday has the wrong key handover detail” lacks a reliable category, customer identifier and urgency, so a model can extract likely entities, propose a category and draft a response while a person confirms before it changes anything.

The same is true for inconsistent documents, long free-text feedback, call transcripts, and knowledge bases where people ask the same question in twenty ways; the useful output is a bounded proposal: reviewable fields, source-passage shortlist, suggested route, draft reply or confidence signal.

For high-impact actions, the model is not final authority: do not let it issue refunds, change payment details, delete records, grant access or send legal representations from a prompt and a shrug. It can prepare the action, but normal permissions, rules and audit trail decide whether it happens.

A good first model-assisted workflow has a narrow schema and honest failure mode: a support triage service can return one of six categories, suggested priority, quoted source line and needs_review: true. If it cannot cite relevant text or confidence is below a tested threshold, it uses the ordinary queue rather than inventing an answer because “unknown” feels awkward.

Separate retrieval from answer writing: a knowledge assistant that searches approved documents, exposes its snippets and declines without a source is useful, while one that blends old policy, a half-remembered support thread and generic web knowledge is a liability dressed as speed.

An AI-automation proposal should include authentication, source permissions and freshness, event logging, rate limits, human review and correction. The model call is one component, not the system. If your use case is knowledge-heavy, the choice between retrieval and fine-tuning is worth making before preparing data. If it calls an external API, the production integration checklist covers expected controls.

Evals, drift and spend are the actual ongoing AI bill

The first model demo is cheap, but ongoing responsibility is not.

An AI workflow needs a test set of real examples with acceptable results alongside them, not five hand-picked demo prompts. Include short messages, badly scanned documents, contradictory requests, abusive inputs, cases whose answer is “send this to a person,” and cases that caused human mistakes, because that evaluation set shows whether a model change improves quality, a prompt edit harms routing, or a source document breaks retrieval. Memory is poor monitoring because fluent bad answers are easy to accept.

Workflows drift even untouched: customers use new language, policies change, upstream providers alter templates, and model providers revise behaviour or retire versions, so the same prompt can produce a different structure, refusal behaviour or cost profile for months before anyone notices.

Someone must own a review queue for uncertain results, regular samples of completed work, alerts for unusual output rates, and comparison with the evaluation set, while spend needs a boundary too. One email summary is cheap, but a model called for every backfill row can become a bill before anybody notices, so set a daily or monthly budget, log tokens and provider errors, cache genuinely repeated input, and degrade safely. A support queue can fall back to manual routing; an invoice process cannot fall back to guessing.

Before sending text to a provider, decide what may leave your system, what to redact, the retention period and which roles may see result logs, because a model response can contain customer data and must be treated as part of the record rather than harmless debug output.

When a buyer asks for “an AI agent that just handles it,” ask: handles what, with which authority, at what cost, and who checks next month? A clear answer may still lead to a model, but not one used to avoid operational decisions.

Build the smallest mechanism that makes bad work visible

Start before anyone writes a prompt and watch the process long enough to find its trigger: automated prospecting may really be unusable registration data, an assistant may mask missed webhooks, and automatic answers may conceal documentation with no owner and contradictions.

Define an observable target: registrations available by the next business morning; every record shows why it has its priority; a monthly reconciliation sweep finds missed events; low-confidence extraction waits for review rather than creating a record. Then build the deterministic skeleton: store raw input, keep cursors and event IDs, make writes repeatable, replay failures, run reconciliation, and report what did not happen as well as what did.

Only then introduce a model where a rule would be dishonest or unmanageable: structure its output, retain input and output together, sample results, give it a budget and let it abstain. That is faster than explaining why an address was overwritten, lead skipped or answer invented, and leaves something useful if the model later disappears.

For where automation work belongs in a product budget, MVP development cost is a better starting point than a model price list.

I build automation around the failure path first: the missed event, empty gap, replay and report of what remains wrong. For the systems I work on and the rate I charge, see my CV.