3 Ethics, Data Governance & Integrity

Decide what may enter the tool before asking what the tool can do with it.

3.1 A decision before the first prompt

This chapter belongs before the first activity because a useful output does not justify an inappropriate input. Use the fictional pilot pack while learning. Do not substitute a real student, employee, client or research participant to make the exercise more realistic.

Apply the same first-use gate to drafting, simulations, feedback, research and workshops:

Classify the material. Is it public, confidential, personal, sensitive, contract-restricted or otherwise controlled? Who might be identifiable from combinations of details?
Check authority and rights. Are you permitted to process it for this purpose and share it with this service? Possessing a document is not the same as having permission to upload it.
Check the approved account and features. Institutional approval should cover the use, account, data class and enabled features, not just a brand name.
Check handling. Consider retention, training use, access, sharing, external search/integrations and relevant processing locations. Ask the institution’s appropriate adviser where information is unclear.
Choose a route. If the requirements are not met, use synthetic material, an approved alternative or the non-AI version. Do not upload first and seek permission afterwards.

Record the decision briefly, without copying the sensitive information into the record. An approved service may still be inappropriate for a particular dataset or purpose.

3.1.1 What enterprise protection does and does not establish

Enterprise services can offer contractual protections and administrative controls. They do not automatically establish your institution’s permission for every use. Features and connected services can have different boundaries. Microsoft, for example, distinguishes Copilot protections from web-search processing. Check the actual service and configuration. Microsoft privacy and protections.

OAIC recommends due diligence appropriate to an AI product’s intended use, including access, oversight and privacy/security risks. Its guidance also recognises re-identification risk. Removing names is not a universal clearance to use an unapproved tool. OAIC guidance on commercial AI products.

De-identification is a contextual risk decision, not a “find and replace names” exercise. A rare role, incident date, small cohort or distinctive quotation may still identify someone. It also does not remove copyright, contractual or research-ethics restrictions. For classroom practice, a genuinely invented case is usually easier to govern than a disguised real case.

3.2 Keep different data risks separate

A conversation can be retained as a service record without being used for model training. If it enters training, the resulting model is not simply a searchable folder of every document; nevertheless, memorisation and extraction can occur. A retrieval-enabled system can also access stored documents through a separate mechanism. These are different pathways.

Published extraction research shows that some training text can be recovered from language models. That rules out blanket claims of impossibility. It does not show that another user can retrieve any arbitrary one-off upload. Carlini et al., Nasr et al..

Training exclusions address one pathway, not every retention, sharing or security risk. Likewise, similar ideas appearing elsewhere establish neither theft nor independent invention. Teach students to identify the evidence needed for a claim instead of substituting reassurance or suspicion.

Credentials, access tokens and unnecessary personal information do not belong in classroom prompts. If sensitive material is accidentally uploaded, stop the activity, preserve appropriate incident details without further disclosure and follow institutional reporting procedures. Do not ask the model to certify that the information has been deleted.

3.3 Transparent use with real boundaries

Students should know what help is permitted, what must be disclosed and what they must demonstrate independently. Permission is task-specific. An institution may legitimately restrict assistance at a particular assurance point while supporting its use during practice.

A suitable statement might say:

In this activity, use only the supplied fictional pack.
You may rehearse with a partner, the printed exchange
or the approved institutional AI service.

Submit your response and two decision notes. Explain
what assistance you used and which claims you checked.
Do not submit unrelated private conversations.

The changed-case task is completed independently under
the conditions stated in the assessment brief.
Access adjustments are arranged before the task.

Ask if a tool, data source or form of help is unclear.
Institutional academic-integrity procedures apply.

Adapt this to the actual unit policy. Do not claim that disclosure makes every use permissible, or that any uncritical use automatically meets the institution’s definition of misconduct. Poor work and dishonest conduct are different questions.

3.4 Detection and assurance

A detector result is not direct evidence of the student’s learning process. Neither is a long, polished conversation. A student may generate a plausible reflection; a genuine learner may write briefly or struggle to explain under pressure.

The assessment chapter therefore combines selected process evidence with meaningful independent demonstrations. These support a learning judgement but do not remove the need for fair misconduct procedures.

Avoid giving students mixed messages. If the assignment permits extensive assistance, do not later penalise its mere use. If it restricts assistance, explain the restriction and the assurance purpose. Follow-up questions should be relevant, accessible and consistent with published expectations.

3.5 Teach verification without forced opposition

Critical engagement does not mean always finding fault. A learner should be able to accept a correct calculation, reject an unsupported inference and say that there is not enough information to decide.

The pilot’s marketing example illustrates all three. The higher click-through rate can be calculated. The claim of greater profit lacks cost data. A causal claim cannot be established from different audiences and weeks alone. The same response can reasonably contain agreement, rejection and uncertainty.

A model-generated persona is not an independent expert. Agreement between three personas does not create three sources. Check against the supplied records, reliable external material or an authorised person, depending on the question.

3.6 Sensitive scenarios and student wellbeing

Performance plans, discrimination complaints and termination conversations can be useful teaching contexts, but students may have personal experience of them. Provide advance notice, a stop instruction, a neutral route and a debrief.

Do not reward disclosure of private information, emotional conformity or success at making the simulated employee agree. A respectful decision may be to pause, seek independent advice or leave the outcome unresolved.

For the pilot, the invented policy is the exercise criterion. It is not legal advice. If teaching actual Australian workplace bullying law, use the complete applicable definition and scope: health/safety risk and the distinction for reasonable management action matter, as well as repeated unreasonable behaviour. Fair Work Ombudsman guidance. Have an appropriately qualified person review consequential legal teaching.

3.7 Equity is a design requirement

Conversational skill does not erase differences in access, language support, model capability, disability or available time. Provide a route that uses the same learning criteria without a paid subscription or personal account.

When the capability is planning and justification, paper or an accommodated spoken response may work. When the capability is live communication, a written case analysis alone is not an equivalent demonstration: provide an appropriate interaction, such as a structured peer/facilitator role-play.

Do not require every private chat as proof of effort. Ask for bounded extracts and permit working notes where appropriate. Set retention and access rules for submitted evidence through institutional processes.

3.8 A bounded ethics discussion

A fictional recruitment tool ranks applicants with career gaps lower. Three candidates question the decision. The supplied facts do not establish the cause or legal outcome.

Ask learners what evidence would distinguish a relevant job criterion from an unfair proxy, who is authorised to review the system, and what should happen while the concern is investigated. They should not diagnose the algorithm by asking another model whether it is biased.

Speculative systemic risks can be worthwhile extension topics, but should be labelled scenarios rather than predictions. A claimed “95% accurate” system needs a defined population, outcome, error distribution and validation method before that number can inform an ethical decision.

3.9 Activity: make the upload decision

Allow 10 minutes; no AI required. Apply the five-step gate to:

The fictional P1 pack.
A placement client’s spreadsheet with names removed.
A student’s draft containing a personal disclosure.

For each, write what you know, what permission or handling detail is missing and the safe alternative. The first can support a bounded practice activity after the account/use check. The second and third need further authority and risk assessment; removing names or choosing an enterprise logo is insufficient.

Keep your resulting rule beside the first student-facing prompt. Safety advice that appears only in a later appendix will not guide the upload decision.

# Ethics, Data Governance & Integrity {#sec-ethics} > Decide what may enter the tool before asking what the tool can do with it. ## A decision before the first prompt {#sec-data-gate} This chapter belongs before the first activity because a useful output does not justify an inappropriate input. Use the fictional [pilot pack](../appendices/pilot-pack.qmd) while learning. Do not substitute a real student, employee, client or research participant to make the exercise more realistic. Apply the same first-use gate to drafting, simulations, feedback, research and workshops: 1. **Classify the material.** Is it public, confidential, personal, sensitive, contract-restricted or otherwise controlled? Who might be identifiable from combinations of details? 2. **Check authority and rights.** Are you permitted to process it for this purpose and share it with this service? Possessing a document is not the same as having permission to upload it. 3. **Check the approved account and features.** Institutional approval should cover the use, account, data class and enabled features, not just a brand name. 4. **Check handling.** Consider retention, training use, access, sharing, external search/integrations and relevant processing locations. Ask the institution's appropriate adviser where information is unclear. 5. **Choose a route.** If the requirements are not met, use synthetic material, an approved alternative or the non-AI version. Do not upload first and seek permission afterwards. Record the decision briefly, without copying the sensitive information into the record. An approved service may still be inappropriate for a particular dataset or purpose. ### What enterprise protection does and does not establish Enterprise services can offer contractual protections and administrative controls. They do not automatically establish your institution's permission for every use. Features and connected services can have different boundaries. Microsoft, for example, distinguishes Copilot protections from web-search processing. Check the actual service and configuration. [Microsoft privacy and protections](https://learn.microsoft.com/en-us/copilot/privacy-and-protections). OAIC recommends due diligence appropriate to an AI product's intended use, including access, oversight and privacy/security risks. Its guidance also recognises re-identification risk. Removing names is not a universal clearance to use an unapproved tool. [OAIC guidance on commercial AI products](https://www.oaic.gov.au/privacy/privacy-guidance-for-organisations-and-government-agencies/guidance-on-privacy-and-the-use-of-commercially-available-ai-products). De-identification is a contextual risk decision, not a “find and replace names” exercise. A rare role, incident date, small cohort or distinctive quotation may still identify someone. It also does not remove copyright, contractual or research-ethics restrictions. For classroom practice, a genuinely invented case is usually easier to govern than a disguised real case. ## Keep different data risks separate A conversation can be retained as a service record without being used for model training. If it enters training, the resulting model is not simply a searchable folder of every document; nevertheless, memorisation and extraction can occur. A retrieval-enabled system can also access stored documents through a separate mechanism. These are different pathways. Published extraction research shows that some training text can be recovered from language models. That rules out blanket claims of impossibility. It does not show that another user can retrieve any arbitrary one-off upload. [Carlini et al.](https://arxiv.org/abs/2012.07805), [Nasr et al.](https://arxiv.org/abs/2311.17035). Training exclusions address one pathway, not every retention, sharing or security risk. Likewise, similar ideas appearing elsewhere establish neither theft nor independent invention. Teach students to identify the evidence needed for a claim instead of substituting reassurance or suspicion. Credentials, access tokens and unnecessary personal information do not belong in classroom prompts. If sensitive material is accidentally uploaded, stop the activity, preserve appropriate incident details without further disclosure and follow institutional reporting procedures. Do not ask the model to certify that the information has been deleted. ## Transparent use with real boundaries {#sec-integrity-principle} Students should know what help is permitted, what must be disclosed and what they must demonstrate independently. Permission is task-specific. An institution may legitimately restrict assistance at a particular assurance point while supporting its use during practice. A suitable statement might say: ```text In this activity, use only the supplied fictional pack. You may rehearse with a partner, the printed exchange or the approved institutional AI service. Submit your response and two decision notes. Explain what assistance you used and which claims you checked. Do not submit unrelated private conversations. The changed-case task is completed independently under the conditions stated in the assessment brief. Access adjustments are arranged before the task. Ask if a tool, data source or form of help is unclear. Institutional academic-integrity procedures apply. ``` Adapt this to the actual unit policy. Do not claim that disclosure makes every use permissible, or that any uncritical use automatically meets the institution's definition of misconduct. Poor work and dishonest conduct are different questions. ## Detection and assurance {#sec-problem-with-detection} A detector result is not direct evidence of the student's learning process. Neither is a long, polished conversation. A student may generate a plausible reflection; a genuine learner may write briefly or struggle to explain under pressure. The [assessment chapter](assessment.qmd) therefore combines selected process evidence with meaningful independent demonstrations. These support a learning judgement but do not remove the need for fair misconduct procedures. Avoid giving students mixed messages. If the assignment permits extensive assistance, do not later penalise its mere use. If it restricts assistance, explain the restriction and the assurance purpose. Follow-up questions should be relevant, accessible and consistent with published expectations. ## Teach verification without forced opposition Critical engagement does not mean always finding fault. A learner should be able to accept a correct calculation, reject an unsupported inference and say that there is not enough information to decide. The pilot's marketing example illustrates all three. The higher click-through rate can be calculated. The claim of greater profit lacks cost data. A causal claim cannot be established from different audiences and weeks alone. The same response can reasonably contain agreement, rejection and uncertainty. A model-generated persona is not an independent expert. Agreement between three personas does not create three sources. Check against the supplied records, reliable external material or an authorised person, depending on the question. ## Sensitive scenarios and student wellbeing Performance plans, discrimination complaints and termination conversations can be useful teaching contexts, but students may have personal experience of them. Provide advance notice, a stop instruction, a neutral route and a debrief. Do not reward disclosure of private information, emotional conformity or success at making the simulated employee agree. A respectful decision may be to pause, seek independent advice or leave the outcome unresolved. For the pilot, the invented policy is the exercise criterion. It is not legal advice. If teaching actual Australian workplace bullying law, use the complete applicable definition and scope: health/safety risk and the distinction for reasonable management action matter, as well as repeated unreasonable behaviour. [Fair Work Ombudsman guidance](https://www.fairwork.gov.au/employment-conditions/bullying-and-harassment). Have an appropriately qualified person review consequential legal teaching. ## Equity is a design requirement Conversational skill does not erase differences in access, language support, model capability, disability or available time. Provide a route that uses the same learning criteria without a paid subscription or personal account. When the capability is planning and justification, paper or an accommodated spoken response may work. When the capability is live communication, a written case analysis alone is not an equivalent demonstration: provide an appropriate interaction, such as a structured peer/facilitator role-play. Do not require every private chat as proof of effort. Ask for bounded extracts and permit working notes where appropriate. Set retention and access rules for submitted evidence through institutional processes. ## A bounded ethics discussion A fictional recruitment tool ranks applicants with career gaps lower. Three candidates question the decision. The supplied facts do not establish the cause or legal outcome. Ask learners what evidence would distinguish a relevant job criterion from an unfair proxy, who is authorised to review the system, and what should happen while the concern is investigated. They should not diagnose the algorithm by asking another model whether it is biased. Speculative systemic risks can be worthwhile extension topics, but should be labelled scenarios rather than predictions. A claimed “95% accurate” system needs a defined population, outcome, error distribution and validation method before that number can inform an ethical decision. ## Activity: make the upload decision **Allow 10 minutes; no AI required.** Apply the five-step gate to: - The fictional P1 pack. - A placement client's spreadsheet with names removed. - A student's draft containing a personal disclosure. For each, write what you know, what permission or handling detail is missing and the safe alternative. The first can support a bounded practice activity after the account/use check. The second and third need further authority and risk assessment; removing names or choosing an enterprise logo is insufficient. Keep your resulting rule beside the first student-facing prompt. Safety advice that appears only in a later appendix will not guide the upload decision.