2 Understanding AI and Large Language Models

A useful explanation of a tool should help you decide how to check its work.

2.1 Model, product and task

A language model generates text from patterns learned during training and the context supplied at use time. Many contemporary models are trained to predict tokens, which may represent words or parts of words, and undergo additional training to improve their behaviour. This mechanism supports useful performance on many tasks; it does not guarantee factual accuracy.

A product may wrap a model with search, document retrieval, calculation, code execution or memory features. Those additions matter. A reply based on a retrieved document has a different evidence trail from an unsupported answer generated from learned patterns, but either can be wrong.

ChatGPT, Claude, Gemini and Copilot are product families, not interchangeable specifications. Features, accounts and institutional approvals change. This book’s workflow does not depend on a particular subscription or interface.

The GPT-4 technical report describes next-token pre-training and subsequent fine-tuning, while withholding model size. Claims that its parameter count is known precisely should not be treated as established facts. GPT-4 technical report, section 2.

2.2 Why fluent answers need checks

A plausible explanation can contain an invented citation, a wrong calculation or a confident inference unsupported by the inputs. Fluency makes the answer easy to read; it is not a confidence interval or a guarantee.

In the pilot’s P1-F feedback, the invented April warning sounds specific. The source card records only a May conversation. Checking the date is a stronger test than asking the same model whether it is certain.

Avoid the opposite overstatement too. “AI cannot reason” is not a useful rule for designing a class. Models can perform tasks involving reasoning, with reliability that varies by task, inputs and system. Observe the actual performance and verify what matters. Neither impressive output nor a blanket dismissal tells you whether the student has learned.

2.3 What a model may help you do

You might ask it to propose alternative classroom questions, play a bounded role, reorganise a draft or suggest explanations at different levels. Each is a candidate use, not an automatic improvement over your current method.

Match the check to the task:

Task	Useful check
Explain a concept	Compare with the taught definition and a counterexample
Summarise a record	Check retained facts, omissions and source locations
Calculate a total	Recalculate from the inputs
Suggest a scenario	Check factual consistency, assumptions and suitability
Give feedback	Compare with a calibrated rubric and specimen responses

A “creative” task can still cause harm through stereotypes, personal data or false claims. A small task can be consequential. Consider the stakes and whether you can detect an error, not only the size or apparent precision of the request.

2.4 Context, retrieval and memory

The active context is the information made available for the current response. It has limits, and a product may truncate, summarise or select parts of a conversation. Even when information fits, the model may not use it reliably.

Some products can retain preferences or retrieve earlier material; others do not. Do not assume either perfect continuity or universal amnesia. For a teaching simulation, the authoritative record should be the source cards and event log, not what a persona seems to remember.

Long-context research found that relevant information’s position can affect performance, sometimes with stronger results near the beginning and end than the middle. That is a warning to test retrieval, not a universal 50-turn rule. Lost in the Middle.

The context chapter shows how to carry a checked brief between sessions. Do not copy unrelated private material just to preserve context.

2.5 Generated perspectives are not independent sources

Three personas can help you imagine competing concerns. They do not represent three people consulted, three experiments or three independent confirmations. They may repeat the same error because they draw on the same inputs and model.

In a business class, ask students to distinguish a possible stakeholder concern from evidence that stakeholders actually hold it. A model’s “employee” can suggest a question to investigate, but cannot establish an employee’s experience.

Likewise, an explanation of how a model reached an answer is not an inspection of its internal computation. Ask for checkable steps, sources, assumptions and calculations. Evaluate those directly.

2.6 The learner remains a separate question

A good output and a learning gain are different outcomes. Assisted practice can improve the work in front of the student without improving their later unaided performance. The design of assistance matters.

One study of high-school mathematics compared ordinary and guardrailed AI support with unaided follow-up performance. It is evidence for checking the difference between assisted performance and learning, not a validation of every AI activity in this book. Bastani et al., author manuscript.

In PDP, the learner therefore attempts a response, examines feedback and then tries a changed case independently. The model is optional in that sequence. The method requires evidence of the learner’s capability, not loyalty to a tool.

2.7 Activity: separate claim from check

Allow eight minutes; no AI required. Read the P1-F feedback in the pilot pack. Underline one factual claim, one judgement and one recommendation. Name the source or criterion needed to check each.

A successful answer checks the April warning against P1-D3, the fairness judgement against P1-D1 and the approval recommendation against P1-D5. “Ask the AI again” may generate another explanation, but is not independent verification.

# Understanding AI and Large Language Models {#sec-understanding} > A useful explanation of a tool should help you decide how to check its work. ## Model, product and task A language model generates text from patterns learned during training and the context supplied at use time. Many contemporary models are trained to predict tokens, which may represent words or parts of words, and undergo additional training to improve their behaviour. This mechanism supports useful performance on many tasks; it does not guarantee factual accuracy. A product may wrap a model with search, document retrieval, calculation, code execution or memory features. Those additions matter. A reply based on a retrieved document has a different evidence trail from an unsupported answer generated from learned patterns, but either can be wrong. ChatGPT, Claude, Gemini and Copilot are product families, not interchangeable specifications. Features, accounts and institutional approvals change. This book's workflow does not depend on a particular subscription or interface. The GPT-4 technical report describes next-token pre-training and subsequent fine-tuning, while withholding model size. Claims that its parameter count is known precisely should not be treated as established facts. [GPT-4 technical report, section 2](https://arxiv.org/pdf/2303.08774). ## Why fluent answers need checks A plausible explanation can contain an invented citation, a wrong calculation or a confident inference unsupported by the inputs. Fluency makes the answer easy to read; it is not a confidence interval or a guarantee. In the pilot's P1-F feedback, the invented April warning sounds specific. The source card records only a May conversation. Checking the date is a stronger test than asking the same model whether it is certain. Avoid the opposite overstatement too. “AI cannot reason” is not a useful rule for designing a class. Models can perform tasks involving reasoning, with reliability that varies by task, inputs and system. Observe the actual performance and verify what matters. Neither impressive output nor a blanket dismissal tells you whether the student has learned. ## What a model may help you do You might ask it to propose alternative classroom questions, play a bounded role, reorganise a draft or suggest explanations at different levels. Each is a candidate use, not an automatic improvement over your current method. Match the check to the task: | Task | Useful check | |---|---| | Explain a concept | Compare with the taught definition and a counterexample | | Summarise a record | Check retained facts, omissions and source locations | | Calculate a total | Recalculate from the inputs | | Suggest a scenario | Check factual consistency, assumptions and suitability | | Give feedback | Compare with a calibrated rubric and specimen responses | A “creative” task can still cause harm through stereotypes, personal data or false claims. A small task can be consequential. Consider the stakes and whether you can detect an error, not only the size or apparent precision of the request. ## Context, retrieval and memory The active context is the information made available for the current response. It has limits, and a product may truncate, summarise or select parts of a conversation. Even when information fits, the model may not use it reliably. Some products can retain preferences or retrieve earlier material; others do not. Do not assume either perfect continuity or universal amnesia. For a teaching simulation, the authoritative record should be the source cards and event log, not what a persona seems to remember. Long-context research found that relevant information's position can affect performance, sometimes with stronger results near the beginning and end than the middle. That is a warning to test retrieval, not a universal 50-turn rule. [Lost in the Middle](https://arxiv.org/abs/2307.03172). The [context chapter](managing-context.qmd) shows how to carry a checked brief between sessions. Do not copy unrelated private material just to preserve context. ## Generated perspectives are not independent sources Three personas can help you imagine competing concerns. They do not represent three people consulted, three experiments or three independent confirmations. They may repeat the same error because they draw on the same inputs and model. In a business class, ask students to distinguish a possible stakeholder concern from evidence that stakeholders actually hold it. A model's “employee” can suggest a question to investigate, but cannot establish an employee's experience. Likewise, an explanation of how a model reached an answer is not an inspection of its internal computation. Ask for checkable steps, sources, assumptions and calculations. Evaluate those directly. ## The learner remains a separate question A good output and a learning gain are different outcomes. Assisted practice can improve the work in front of the student without improving their later unaided performance. The design of assistance matters. One study of high-school mathematics compared ordinary and guardrailed AI support with unaided follow-up performance. It is evidence for checking the difference between assisted performance and learning, not a validation of every AI activity in this book. [Bastani et al., author manuscript](https://hamsabastani.github.io/education_llm.pdf). In PDP, the learner therefore attempts a response, examines feedback and then tries a changed case independently. The model is optional in that sequence. The method requires evidence of the learner's capability, not loyalty to a tool. ## Activity: separate claim from check **Allow eight minutes; no AI required.** Read the P1-F feedback in the [pilot pack](../appendices/pilot-pack.qmd). Underline one factual claim, one judgement and one recommendation. Name the source or criterion needed to check each. A successful answer checks the April warning against P1-D3, the fairness judgement against P1-D1 and the approval recommendation against P1-D5. “Ask the AI again” may generate another explanation, but is not independent verification.