12  What comes next?

Tessera’s board receives an evaluated prototype and a recommendation to revise it. The fault test failed. The reviewer knows why the proposed deployment should not proceed, what a safer next experiment would require, and who must approve it.

That is useful work even without a launch. The final task is to carry the evidence into decisions about people, resources, and continued operation. A technically sound comparison answers only part of that question.

12.1 The efficiency claim needs a denominator

“Five times faster” might describe drafting a paragraph, not completing a support case. A fuller measure includes preparation, verification, correction, escalation, and the work created elsewhere. Measure comparable tasks at an agreed quality level and keep failures in the sample.

For a constructed example, the old process takes 12 minutes per ticket. A new tool drafts in 2 minutes but requires 5 minutes of review and 3 minutes of correction: 10 minutes in total. That is a two-minute saving, not a sixfold improvement. At 100 tickets, the arithmetic suggests 200 minutes saved before training, setup, and incident costs. It does not prove the result will persist at a different workload.

Record whose time changed. A saving for the author can become extra work for a supervisor or customer. If reviewers cannot maintain the required standard at the proposed volume, the checkpoint design has failed even if the model is fast.

12.2 Tasks, roles, and uncertainty

A job contains tasks, relationships, responsibilities, and institutional knowledge. Automating one task can change the others. It may reduce demand for a role, expand a service, or shift work towards review. None of those outcomes follows inevitably from a faster draft.

Acemoglu’s task-based economic analysis connects aggregate effects to which tasks are affected and their productivity gains. Its estimates depend on assumptions; a macroeconomic model is not a forecast for Tessera’s staffing. Use it to ask better questions about scope and distribution, not to attach a precise employment prediction to a small prototype. The Simple Macroeconomics of AI

Lower costs may increase demand. They may also leave demand unchanged, or savings may be absorbed by constraints elsewhere. A rebound effect is a possibility to model, not proof that cheaper AI always expands work. Avoid using “Jevons paradox” as a substitute for measuring the organisation’s backlog, budget, and demand.

12.3 Who benefits, who pays, who is missing?

Keep three lists in the handover.

Benefits: which users receive a faster or better service, and how will you measure that improvement? Include accessibility, consistency, and the ability to resolve an issue, not just throughput.

Transition costs: who learns the new workflow, corrects its errors, maintains its sources, and handles exceptions? Make those hours visible. A team member moved into reviewing opaque outputs may need training, time, and a route to refuse unsafe work.

Missing experience: which customer groups, languages, devices, or situations were not represented in evaluation? Missing evidence is not evidence of harm, but neither is it permission to claim equal performance. Plan how those experiences will be assessed before expanding use.

The label distributions in the customer lab cannot answer these questions by themselves. In particular, they do not prove that a group was excluded from training or that it receives more wrong predictions. The chapter on For whom? established the evidence needed for that comparison; keep that distinction through the final pitch.

12.4 A second-order memo with honest limits

Suppose the prototype suggests shorter handling time on a small synthetic exercise. A weak memo says, “Automate support and redeploy the savings.” A stronger one says:

We have demonstrated the workflow on invented cases, not measured customer outcomes. A bounded pilot would need approved data, representative tickets, reviewer training, a manual fallback, and a defined duration. Measure total handling time and missed issues by relevant group. Staff changes are outside the evidence available from this prototype.

That version gives a decision-maker something to approve or reject. It also prevents a teaching exercise from quietly becoming an operational claim.

For your capstone, choose an owner for each unresolved measurement. Do not assign every item to “the team.” If nobody can obtain the evidence, the proposed next stage is not yet planned.

12.5 Maintenance is part of the argument

A model update, a changed prompt, a revised runbook, or a new customer population can alter performance. Preserve the previous version and results. Run a controlled comparison before replacing a load-bearing component, and decide what would trigger a rollback.

A practical review record has a date, version, sample definition, per-class or per-group results, incident summary, decision, and named owner. Distinguish regular monitoring from a new acceptance evaluation. Monitoring may identify a problem; it does not automatically explain its cause.

The stop rule must be usable. If a reviewer sees unsupported recommendations or a missed-fault threshold is breached, who can disable the affected action? Where do pending cases go? Can the ordinary support process absorb them? Rehearse the fallback before it is needed.

Appendix D supplies the stage gates so that the next decision depends on explicit evidence.

12.6 Plan for several futures

Instead of predicting a single capability trajectory, consider three scenarios: improvement slows, improvement continues, or a particular capability changes sharply. Which parts of your workflow remain useful in each?

Portable assets include clear task definitions, test cases, source records, output validators, access boundaries, and the ability to compare a replacement with the current baseline. A prompt tuned to one interface may be less portable. A team that can explain its acceptance rule is better placed to revise it deliberately when the use case changes.

Do not promise that human jobs or particular skills are permanently protected. The narrower commitment is within your control: keep responsibility explicit, practise checking work, and retain evidence that lets another person challenge a decision.

12.7 Independent capstone: defend the decision

Time: 45–60 minutes. Inputs: your original proof brief, the evidence dossier, and the benefits, transition costs, and missing-experience analysis from this chapter. No model is required. If you followed the paper route, use the supplied router results or Appendix G’s lab and label the whole submission a case analysis, not a completed build.

Write a two-page decision memo for Tessera’s board:

1. State the intended use and forbidden use. Show the original acceptance rules beside the actual evidence; explain any change to the brief.
2. Compare with a baseline or explain why that comparison remains untested. Include counts and subgroup or condition failures, not just favourable averages.
3. Name who benefits, who bears mistakes or transition costs, and whose experience has not been measured. Separate evidence from forecasts.
4. Recommend go to a bounded pilot, revise and retest, or stop. Specify the human checkpoint, stop condition, accountable role, and next review.

Then read it as a sceptical colleague. Find one sentence that claims more than the evidence permits and rewrite it. An optional AI critique comes after this independent review.

Success: another reader can trace the decision to the evidence and identify what remains unknown. Use the acceptance checklist in Appendix C and the stage gates in Appendix D. An honest stop decision can meet every criterion; an impressive demo without the record cannot.