10 How does it decide?
Rule B scores three out of four on the lab’s test rows; Rule A scores two out of four. B is easy to explain. It still misses one of the two Low-labelled cases, so neither rule meets the brief.
That is a model-selection decision you can defend. You know the rules, the inputs, the errors, and the acceptance criteria. A more complex comparison should preserve those same ingredients, even when the model’s internal behaviour is harder to inspect.
10.1 Predictive rules are not causal explanations
A small decision tree exposes a path of conditions leading to a prediction. A logistic regression combines features into a score that is transformed into a probability; it is linear in the log-odds with respect to the supplied features, not necessarily in the resulting probability. Transformations and interactions can extend those features.
Coefficients need interpretation in context. Their signs and magnitudes depend on coding, scale, correlated variables, regularisation, and the data used to fit them. The largest coefficient is not automatically the strongest business driver, and a coefficient is not evidence that changing that feature causes the outcome.
A neural network combines learned transformations across layers. Its internal reasoning is generally harder to summarise faithfully than a short tree, but “black box” does not mean nothing can be tested or explained. You can inspect inputs, outputs, error patterns, sensitivity, and the limitations of an explanation method.
Nor is every tree simple. A very deep tree can be difficult to follow. A random forest combines many trees and sacrifices some direct traceability. Choose an explanation requirement for the actual decision: tracing a prediction, explaining a general pattern, and establishing the effect of an intervention are different jobs.
These distinctions are developed in the authors’ Introduction to Statistical Learning. The relevant lesson here is practical: name what your explanation shows and what it does not.
10.2 Compare on the same question
A fair comparison needs a shared target definition, test inputs, acceptance rules, and accounting of costs. A classifier that predicts a Low-renewal label cannot be compared as though it had predicted actual departures. A language model’s elegant explanation is not a substitute for its recorded predictions.
Use development data to choose features, tune settings, and investigate failures. Reserve a separate evaluation for the claim you intend to make. Where customers recur across records, or the task predicts future events, the split must respect those dependencies. Randomly splitting rows can leak information across the boundary.
Reusing a held-out set until a candidate wins makes it part of development. Keep that history and obtain fresh evidence for the final claim. The same applies to prompts: repeated revisions against the same questions are model development even if no weights change. scikit-learn: data leakage and common pitfalls
10.3 Tessera: the comparison sheet
Use Appendix G’s four test rows and predefined rules. The predictions are printed so that anyone can reproduce the decision without installing a tool.
| Candidate | Correct / tested | Low cases detected | False Low alarms | Meets the brief? |
|---|---|---|---|---|
| Rule A: satisfaction ≤ 6 | 2/4 | 1/2 | 1 | No |
| Rule B: tenure ≤ 6 | 3/4 | 1/2 | 0 | No |
| Always High | 2/4 | 0/2 | 0 | No |
A manager might reasonably ask whether B is worth further development. It is a candidate for another test, not an approved solution. Its higher accuracy does not override the predeclared requirement to detect both Low-labelled cases.
Constructed review: an AI drafts “Choose B: best accuracy and no false alarms.” The reviewer adds the missing column: one Low case is missed. The corrected memo recommends further investigation without changing the threshold after seeing the result.
Even a perfect score on four rows would not establish a dependable real-world rate. The small sample exists to expose the logic of the decision. The dossier should distinguish that teaching result from any claim about future customers.
10.4 When a smaller model is enough
Model size, deployment location, precision format, and task performance are separate variables. A compact model may be sufficient for a narrow task and fit a useful operating budget. It may also fail a critical case that a larger model handles, or run too slowly on the available hardware. Measure rather than infer a winner from the label.
Quantisation represents numerical values with reduced precision. It can reduce memory requirements; its effect on speed and quality depends on the method, runtime, hardware, and task. A model file’s size is not the total memory required to run it, especially with a long context or concurrent requests. The experimental literature on post-training quantisation illustrates these trade-offs; it does not promise one universal result. GPTQ, Frantar et al.
If you compare quantised variants, keep the task and scoring rules fixed. Record the exact model identifiers and settings. Include latency, resource use, and failures alongside quality, and preserve the data-boundary requirements established earlier.
10.5 Provenance and synthetic training data
Synthetic data is not automatically bad training data. The question is how it was produced, selected, mixed, and evaluated. Shumailov and colleagues demonstrated degradation under recursive model-generated training in the regimes they studied, including loss of distributional tails. That supports a concern about careless replacement of data, not a theorem that every use of synthetic examples ruins a model. Shumailov et al., Nature, 2024
Training degradation is also not the same mechanism as a retrieval assistant reading a wrong document. For retrieval, check source quality, access, currency, coverage, and whether the cited passage supports the answer. Human authorship alone guarantees none of those.
Keep provenance for both. The lab-v1 fixtures are safe to inspect because they are explicitly invented, not because they represent a population or carry validated outcomes. They teach evaluation; they do not train away the need for it.
10.6 Compare retrieval without inventing the internals
A product such as NotebookLM can be an optional comparison for the source-grounded exercise. You can observe its answers and supporting passages. You cannot infer a specific hidden chunking or ranking algorithm merely because the interface looks polished.
Use the same saved sources and questions for each candidate. Appendix G provides an answerable fault-review question, a question that requires preserving a limitation, and a refund-amount question that the sources cannot answer. Score support for each consequential claim and the handling of missing evidence.
A design sketch such as ingest, chunk, embed, retrieve, rank, generate, cite is a useful hypothesis about possible components. It is not a verified account of every retrieval system. The original retrieval-augmented generation work describes one architecture, not all products sold under the label. Lewis et al., 2020
10.7 Core: choose neither when necessary
Time: 30 minutes. Inputs: Appendix G’s customer rows, rules, and acceptance criterion.
- Reproduce the three rows of the comparison table.
- Trace one error from its inputs through each candidate’s rule.
- Write what that trace explains, and why it establishes no causal effect.
- Recommend select, revise, or stop against the original criterion.
- Specify a fresh test needed after any revision.
Artefact: a comparison sheet and short decision in the dossier. Success: preserve both denominators, reject all three candidates under the supplied brief, and separate “best among these” from “good enough.”
Optional build: compare fitted models with an approved tool, retaining the same splits and target definition. Report what actually happened, including a different ranking. Do not invent a confusion matrix for a model you did not run. For the retrieval alternative, use the printed source-answer key and label any unexecuted candidate “not tested.”
Next, the agent chapter adds permissions and actions. A selected component is only one part of that system; its comparison does not authorise unattended execution.