10 Assessing Learning: Process, Product and Independent Capability
A record of work gives you something to examine. It does not tell you everything about the person who submitted it.
10.1 Begin with the capability
A negotiation, a financial analysis and an evidence-based recommendation require different assessment evidence. Start by naming what the student must be able to do, then decide which parts may be supported and which need an independent demonstration. AI literacy can itself be an outcome, but it should not silently replace the disciplinary outcome.
In the pilot pack, the capability is to plan and justify a fair first response. The learner must distinguish known facts, uncertainty and authority. That is narrower than conducting a full workplace investigation. An articulate chat transcript cannot establish the latter.
Process and product both matter. A report that sounds thoughtful but contains an unsafe recommendation is deficient. A correct answer reached through a method the learner cannot explain may leave an assurance gap. Neither concern requires treating all AI use as misconduct.
10.2 Three kinds of evidence
Supported practice gives students feedback while they learn. Prompts, peers, worked examples and models may help. Do not treat the resulting performance as unaided capability.
Submitted process evidence makes selected decisions inspectable: what the learner considered, checked, changed or retained. It can support assessment and feedback, but transcripts, journals and reflections can be reconstructed or fabricated. Length and apparent authenticity do not make them conclusive.
Independent demonstration asks the student to apply relevant knowledge under conditions appropriate to the outcome. It might be a supervised changed-case response, an observed practical task or an accessible oral explanation with follow-up questions. One short check has limited coverage; assurance generally draws on several meaningful points across a unit or programme.
The TEQSA-commissioned 2025 resource describes programme-level, unit-level and hybrid approaches. Its advice supports contextual, inclusive assurance, not a single universal format or prescribed marks split. Enacting assessment reform.
10.3 A worked design decision
Suppose students submit P1-A from the pilot pack, with a long transcript showing several rounds of “improvement”. The response invents a warning and approves an arrangement beyond the adviser’s authority. A marker should score those defects against the task criteria. The number of turns earns no compensating credit.
P1-B is shorter and could have been written without AI. It identifies missing evidence, preserves a response opportunity and refers a decision appropriately. Those observable qualities merit credit. A marker need not infer that every sentence reveals the writer’s internal thinking.
The independent P2 task changes the situation: the complaint concerns possible retaliation by the manager. A competent learner should recognise the need for independent review. Repeating the P1 response without addressing that conflict does not satisfy the changed task.
This three-part design gives the educator more useful evidence than a final submission alone: a practice response, a checked explanation of a revision and a new application. It still does not prove lasting competence after one lesson.
10.4 Three assessment models
10.4.1 Model 1: simulation and process review
Students rehearse a bounded professional interaction and select two moments to analyse. For each, they explain the decision, the relevant criterion and what the available evidence supports. They may identify a sound decision, not only mistakes.
Use the conversation to assess what is visible. A typed simulation may show question selection and written communication; it cannot establish performance in a real-time spoken encounter. If live communication is an outcome, include an observed interaction and appropriate access adjustments.
10.4.2 Model 2: evidence-based recommendation
Supply the data and constraints. Students examine a draft recommendation, test its calculations and assumptions, then justify their own recommendation. The supplier task S1 gives a complete small example: the cheapest quote cannot meet the deadline quantity.
AI can propose alternatives, but it cannot create a missing permission, invent capacity or turn an unsupported claim into a fact. Require references to the supplied records and a clear statement of what remains unknown.
10.4.3 Model 3: feedback evaluation
Give students a response, criteria and deliberately flawed feedback. Ask them to accept, correct or withhold judgement on each feedback claim. They must explain the basis of their choice.
Do not require two overrides regardless of what the feedback says. That rewards disagreement as performance. When error detection is the intended skill, use a checked specimen containing known defects and explain the answer criteria.
10.5 Make process evidence manageable
Use a short evidence index instead of requiring every private conversation.
| Decision | Evidence/check | Result | Consequence |
|---|---|---|---|
| Is the plan already decided? | P1-D1 and D5 | Proposal; approval not given | Do not announce a final decision |
| Was there an April warning? | P1-D3 | Only May contact recorded | Reject invented warning |
| Can I approve later starts? | P1-D5 | Adviser lacks authority | Refer request |
A learner can attach a brief draft extract, working note, calculation or conversation passage. The marker reads the reasoning and checks the cited material. An index is not proof that the learner performed the work; it is a way to locate assessable evidence efficiently.
Do not map Flesch scores, prompt length or turn counts to achievement bands. They measure features of text or activity, not the learning outcome. Efficient work, assistive tools, language choices and task design can change those measures. This book does not supply or recommend an automated engagement classifier.
10.6 Set weights deliberately
The pilot uses five equally weighted criteria, each scored 0–4: evidence, process, judgement, communication and verification. The rubric appendix explains the descriptors.
For another task, a different balance may be appropriate. Two illustrative allocations show why:
| Criterion | Evidence-based report | Observed consultation |
|---|---|---|
| Disciplinary accuracy | 30% | 20% |
| Method and evidence | 30% | 25% |
| Justified decisions | 20% | 20% |
| Communication | 10% | 25% |
| Limits and verification | 10% | 10% |
| Total | 100% | 100% |
These are design examples, not validated instruments. Specify where each criterion is observed and avoid rewarding the same action twice. A requirement to explain a decision should not also become a separate reward for producing a long transcript.
Students who do not use AI can submit working notes and equivalent evidence. If appropriate use of AI is itself an assessed outcome, explain that requirement, provide approved equitable access and agree accommodations through institutional processes. Do not disguise a compulsory AI task as an optional route.
10.8 Fair follow-up and workload
Tell students in advance what independent checks involve, their purpose, permitted resources, criteria and access arrangements. Ask relevant questions at a comparable level of difficulty. Avoid a surprise interrogation framed as a friendly conversation.
An unexplained error, weak oral answer or sudden improvement warrants careful follow-up, not an automatic misconduct finding. Assess the demonstrated work against the published criteria. If misconduct is suspected, use the institution’s evidence, notice, response and appeal procedures. Anxiety, disability and language factors can affect performance and should not become proxies for dishonesty.
Budget the work. Five minutes for each of 100 students is 500 minutes, or 8 hours 20 minutes, before scheduling, documentation, moderation and accommodations. A team might distribute checks across tutorials or choose a different assurance design. Do not promise scalability without those costs.
10.9 Design activity: an assurance map
Allow 15 minutes; no AI required. Choose one assessment and make four entries:
- The capability, expressed as an observable action.
- What supported practice prepares the learner.
- What submitted evidence you will inspect.
- Where independent capability is demonstrated and what remains untested.
Then use the stress-test sequence. A sound map permits appropriate help while making the remaining assurance gap explicit. If the map only says “submit the transcript”, it needs another design decision before release.