Appendix E — A Calibrated Assessment Rubric
A rubric should describe the capability being assessed and the evidence needed to judge it. This appendix provides a teaching template, not a validated instrument or a universal grading scheme.
The pilot pack includes responses, source cards, example scores and a completed correction. Read them together. A table of adjectives alone does not establish what a particular level looks like.
E.1 The pilot criteria
Each criterion is scored 0–4 and carries 20% of the total. Five criteria give a maximum of 20 points. The same criteria apply to paper, typed and appropriately accommodated spoken responses to the pilot’s planning-and-justification task.
E.1.1 Evidence and accuracy
4: All consequential claims are supported by the supplied record; facts, allegations and missing information are distinguished precisely.
3: Main claims are supported; a limited omission does not change the decision.
2: Some correct evidence is used, but a material uncertainty is missed.
1: A relevant concern is recognised, but important claims are unsupported.
0: The response’s factual basis is absent or fundamentally contradicted.
E.1.2 Fair process
4: Gives a meaningful response opportunity, avoids premature findings and uses the required independent route when relevant.
3: Generally fair process with a limited omission.
2: Basic response opportunity is present, but the process needs clarification.
1: Acknowledges fairness while proposing a materially deficient process.
0: Predetermines the outcome or removes the required opportunity or safeguard.
E.1.4 Communication
4: Clear, respectful, specific and suited to the task; invites clarification without coercion or unnecessary disclosure.
3: Clear and respectful, with a limited ambiguity.
2: Understandable and partly appropriate, but important wording is unclear or presses for agreement.
1: Meaning is difficult to follow or the approach obstructs a fair exchange.
0: No usable response to the communication task.
E.1.5 Verification and limits
4: Identifies the checks that could change the decision, avoids unsupported assurances and states the limits of the conclusion.
3: Relevant checks and limits are identified, with a minor omission.
2: Recognises uncertainty but gives only a partial verification plan.
1: Makes a general promise to check without identifying the material gap.
0: Treats unverified assumptions as settled or offers unsupported certainty.
Do not award marks for verbosity, number of prompts, eye contact, confidence or the simulated employee’s agreement. These are not proxies for the pilot’s target capability.
E.2 Calibration before use
Score P1-A and P1-B independently before reading the pack’s educator annotations. Then discuss disagreements with a colleague.
| Criterion | P1-A | P1-B |
|---|---|---|
| Evidence and accuracy | 1 | 3 |
| Fair process | 0 | 4 |
| Judgement and authority | 1 | 4 |
| Communication | 2 | 3 |
| Verification and limits | 0 | 3 |
| Total /20 | 4 | 17 |
These are authored teaching scores, not trial results. P1-A’s polished tone does not compensate for an invented warning and authority overreach. P1-B earns more through evidence and restraint, though its verification next step could be more specific.
For practice, the pack proposes at least 14/20, with Fair Process and Judgement each at least 2/4. An institution must approve its own assessment standard. Do not import the threshold or label it a universal pass mark.
If two markers disagree, identify the exact response passage and descriptor. Clarify the rubric or accept a justified range where the evidence permits one. A model agreeing with one marker does not settle the dispute.
E.3 Process evidence without a transcript requirement
A learner may provide a decision note, annotated source, calculation or selected conversation extract. Each should make a consequential decision inspectable.
| Evidence item | What to examine |
|---|---|
| Decision note | Rationale, supporting record and remaining uncertainty |
| Calculation | Inputs, method, result and constraints |
| Revision extract | What changed or stayed and why |
| Independent task | Application to a changed condition |
A long conversation is not automatically strong evidence. A short response is not automatically delegation. Do not map turn counts or readability scores to grade bands, or use them to infer misconduct. Submitted process evidence can be reconstructed; use the wider assurance design described in the assessment chapter.
E.4 Adapt to the outcome
For the supplier example, replace Fair Process with Constraint Handling. Assess whether the decision meets quantity, deadline and budget. For marketing, replace it with Comparison Design: the learner must distinguish a measured association from a causal or profitability claim.
For an observed consultation, explicitly assess the relevant interaction skill. Do not claim that a written plan alone demonstrates it. Plan accessible alternatives that preserve the capability rather than simply changing the topic.
If AI literacy itself is an outcome, add a criterion for appropriate use and verification, explain its weight and provide equitable approved access. Otherwise, do not reserve high marks for students who chose to use AI.
E.4.1 An audit-context adaptation
An information-security audit task would need evidence quality, criteria, testing, findings and justified conclusions. Interviews with simulated employees can support practice, but agreement among personas is not corroboration. A strong report might conclude that evidence is insufficient.
Use an original supplied case or authorised resources. This book does not require a CloudCore service or another book’s live companion site. Consult Substantiate, Don’t Assume for the fuller audit-learning route rather than treating this short rubric adaptation as a complete audit curriculum.
E.5 Build a complete marking scheme
Specify all weights so they total 100%, identify where each criterion is observed and explain any critical requirements. Avoid double-counting a single revision as inquiry, conversation quality and reflection.
Use the assessment chapter’s report/consultation allocations as examples only. A unit may weight disciplinary correctness more heavily than communication, or require a minimum safe-performance standard. State that before students start.
Apply the draft rubric to contrasting work: concise competent, verbose weak, partly correct and insufficient-evidence responses. Check that the resulting marks match the capability you meant to assess.
E.6 Activity: moderate a disagreement
Allow 15 minutes; no AI required. Score P1-A without looking at its annotations. Compare with a colleague or the supplied scoring rationale. Identify one disagreement and cite the descriptor and source card that would resolve it.
Then change the format: imagine the identical reasoning submitted as a short spoken response with pauses. Would you mark the reasoning differently? If so, identify whether the changed feature is genuinely part of the outcome or an unrelated presentation preference.
E.7 Before release
Check that students receive the rubric, permitted-help rules, task conditions, data-handling instructions and access arrangements before beginning. Retain a moderated example and record the rubric version.
An assessment instrument improves through use and review. The printed template is a starting point, not evidence that your local marking is reliable.
For research on rubric design, see Dawson’s Assessment rubrics. For the broader assurance approach, see TEQSA’s 2025 assessment resource.