2  For whom?

A proposed retention model sends a list of customers to a support team. Before asking whether its score is impressive, ask who receives a call, who is missed, and what happens in either case. Those questions turn “works well” into an evaluation plan.

The preceding chapter separated a rule, a model, and an action-taking system. This chapter adds the people affected by each. You do not need a running model to identify the intended population and the evidence you would need before acting.

2.1 An overall score leaves people out of view

Suppose a test contains many examples from one region and few from another. A high overall score can coexist with poor results in the smaller group. It can also coexist with good results there. The aggregate alone does not tell you which.

Request the counts behind the score: examples tested, correct predictions, false alarms, and missed cases for the groups relevant to the use. Include a reason for each grouping. Region may matter because the service or data collection differs; it is not automatically a proxy for a protected characteristic or a complete fairness analysis.

Small groups need careful interpretation. A perfect result on three examples does not establish dependable performance. Nor does a disparity by itself explain its cause. Preserve the uncertainty and ask what additional evidence would distinguish possible explanations.

2.2 Context changes the task

A tool used in different places may face different languages, infrastructure, expectations, and legal obligations. Do not turn a country’s name into a single cultural profile. Ask the intended users how the workflow fits their situation.

Language performance is one reason to test locally. Dey and colleagues compared several models using English and South Asian language settings, finding better English-prompt results in their evaluation. That is evidence about those models, prompts, and tasks, not a permanent ranking of languages or every current service. Dey et al. (2024), Better to Ask in English

For an actual rollout, make the requirements concrete:

• Can users read, hear, or otherwise access the output?
• Are examples and terminology appropriate to the task?
• Can the workflow operate with the available connectivity and devices?
• Which permissions and obligations apply to this use?
• Can someone reject or challenge a consequential result?

Legal requirements vary by jurisdiction and use, and can change. Chapters 5–6 introduce data boundaries and procurement questions; they do not provide a country-by-country compliance approval. Obtain qualified review where required.

2.3 Access within the team

Imagine a class or team in which some people have paid accounts and others do not. A required feature that only some can access changes what work they can complete. A shared environment or a paper route can reduce that barrier.

A local model is one option, not an automatic equity solution. Hardware, installation rights, accessibility, support, and time can also exclude people. A shared hosted system may help, but requires permission and data-handling review. Compare the whole arrangement, not just the subscription price.

This edition’s core exercises use printed inputs so that completing the dossier does not depend on a particular account. Optional builds should record the environment and any obstacles instead of treating setup success as evidence of analytical ability.

2.4 Whose objective is being optimised?

A retention intervention may benefit the business while inconveniencing a customer. A missed call and an unnecessary call have different costs. Ask affected people what matters, not only the team paying for the tool.

There may be legitimate disagreement. One stakeholder values speed; another values avoiding unwanted contact; another needs an explanation and a way to correct a record. These interests cannot all be collapsed into overall accuracy.

Gabriel’s philosophical analysis distinguishes alignment with instructions, preferences, interests, and values, and considers fair principles amid disagreement. It helps frame the discussion; it does not supply a universal numerical fairness threshold. Gabriel (2020), Artificial Intelligence, Values and Alignment

Commercial pressure can encourage premature release, but it does not make failure inevitable or caution commercially pointless. In your proof brief, name the deadline pressure and the approval that remains necessary. If the evidence will not be ready in time, reduce scope, defer, or stop rather than quietly changing the acceptance rule.

2.5 The Tessera label is not an outcome

Appendix G contains twelve invented customer rows. Six have a Low renewal-likelihood label. South has four of five, Metro two of four, and North none of three.

Those figures describe labels. They do not establish actual churn, predictive errors, discrimination, or exclusion from the training data. A region with many Low labels could be easy or difficult for a particular rule to predict.

A constructed conversation illustrates the mistake:

Draft analysis: “South has the highest Low-label rate, so the model performs worst for South.”

Human correction: “Show predicted labels alongside independently established outcomes. The table supplies neither model errors nor actual churn.”

Revised conclusion: “The label distribution differs. Regional predictive performance and its causes are not established.”

Keep both the distribution and the unanswered question. Erasing the unsupported verdict should not erase the reason to investigate.

2.6 Place the decision, not the demographic label

A draft stakeholder map needs checking for omissions and unsupported claims. A decision that changes a person’s treatment needs the relevant evidence, approval, and means of correction. Merely mentioning a group does not establish either safety or harm.

Use Appendix A’s precision-and-consequence questions for the intended action. For the retention example, specify who reviews the list, what records they check, and what happens when an outcome or source is missing.

The next chapter turns this stakeholder map into a four-lens audit. First, complete the map without pretending the supplied data answers more than it does.

2.7 Core: map the people before the score

Time: 20 minutes. Inputs: Appendix G’s customer table and a proposed use: decide which customers receive a retention call.

List the groups represented, the people absent from the table, and the costs of a missed call and an unnecessary call. Calculate Low-label prevalence by region. Then write the different evidence needed to evaluate prediction errors: predictions, independently established outcomes, dates, and group counts.

Artefact: a stakeholder/evidence map. Success: show South 4/5, Metro 2/4, North 0/3 as label distributions, not fairness verdicts. State that actual churn and who was excluded from training are unknown.