TFIS · AI Ethics in Education · Course

Understanding AI Ethics
in Student Learning

Principles, Risks and Responsibilities

Session 2 · Tuesday 1 September 2026 · 8.00–10.00 pm

Evening rule: no lecture block over 12 minutesTonight is triage training, not a philosophy seminar

Re-entry

SAY "This afternoon I asked you to write down one sentence: we are running a natural experiment on our students without a control group. Ethics is what you do when you notice that sentence and refuse to look away. By ten o'clock you will be able to look at any AI-in-learning situation and answer three questions fast: which principle is under pressure, how severe is the risk, and whose job is it."

A true story · no frameworks yet

England,
13 August 2020

Exams cancelled by the pandemic. An algorithm awards the grades instead.

The Ofqual algorithm · what happened

Teachers predicted. The algorithm "corrected."

Teachers submitted assessed grades and rankings. The algorithm adjusted them — primarily against each school's historical results.
On results day, ~39% of grades came out lower than the teacher assessment.
Large state-school cohorts absorbed the downgrades. Small classes — typical of private schools — defaulted to teacher grades. A/A* rose fastest at independent schools.
The canonical victim: predicted A at a historically weak school → awarded C → university place lost.

Case dossier: sources/cases/ofqual-2020.md

The Ofqual algorithm · the collapse

4 days

from results day to full government U-turn (17 August): grades reverted to teacher assessment

senior officials subsequently left their posts

The algorithm was defensible as statistics — it was built to prevent grade inflation, and it did. It was indefensible as a way to treat a student.

Work the case · tables · 10 min

No frameworks. Just answer:

What exactly went wrong? (Not "the algorithm" — what about it?)
Who was harmed — and was the harm distributed fairly?
Who was responsible? Who was held responsible?

Harvest answers on the board — keep their exact words. They come back in ten minutes.

Before you blame the pandemic — or the technology

St George's Medical School, London — the 1980s

Admissions screening automated with a program written to replicate the human selection panel. It replicated them faithfully — including their biases.
It systematically deducted chances from women and applicants with non-European names. Invisible to applicants. Ran for years.
1988: the UK Commission for Racial Equality found the school guilty of discrimination.

Same failure as Ofqual. Forty years apart. No LLMs involved in either.

Case dossier: sources/cases/st-georges-1980s.md

What both cases teach before any framework

Two lessons that survive to 2026

Automation launders bias into consistency. The St George's program applied the same prejudice to everyone, every time. Consistency is not fairness — it is bias at scale with an audit-proof face.
"The computer decided" diffuses accountability. Nobody owned the discrimination because nobody performed it. The machine did — and the machine was doing what the institution had always done.

A modern LLM absorbs bias from training data the way that program absorbed it from panel history. The 1980s version needed someone to code the bias. The 2020s version comes pre-loaded.

Now — and only now — the framework

You already named the principles.

Look at the board. You said: students were graded on their school's past, not their work · nobody could see or contest the mechanism · real damage, unequally spread · built to defend a statistic, not students · everyone pointed elsewhere.

Oxford's AI4People synthesis distilled dozens of AI-ethics frameworks into five principles. They are labels for what you just said. You didn't need the framework to see the failure — the framework is a memory aid.

Floridi et al., Minds and Machines (2018)

Five principles · one slide · the lens for three days

Principle	The test
Beneficence	Does this use make the student more capable next month?
Non-maleficence	No harm — including invisible harm: deskilling, false accusation, privacy leakage.
Autonomy	Students choose how they learn; educators choose how they teach. Dependence is autonomy decay on an installment plan.
Justice	Fair distribution of benefit and harm: access gaps, biased tools, unequal accusation rates.
Explicability	The AI-specific addition: if you can't explain how AI touched a grade, you can't defend the grade.

Floridi et al. (2018). Stanford HAI's founding premise — Fei-Fei Li: "a historical opportunity and responsibility to establish a human-centered framework."

Rule of this course

Never assert bias.
Produce it.

Live demo · P03 · ~15 min

Two reference letters, identical achievements

Chat 1: "Write a 150-word academic reference letter for Ahmad, final-year student: CGPA 3.7, led the student chapter, co-authored one conference paper."

Chat 2 (new chat): identical prompt — the only change is the name: Aisyah.

Room task: hunt the adjective delta. Research pattern (Wan et al., 2023): agentic language for men ("leader," "exceptional"), communal language for women ("warm," "supportive").

Wan et al., EMNLP Findings 2023

Sometimes the live run comes back clean

SAY · if clean "Good — the vendors patched the famous one. The lesson survives: bias in these systems is an empirical property that shifts with every model version. You don't audit once; you audit per model, per use. That is Discernment, and we train it tomorrow."

Bring it home

Trained on whose world?

These models are trained predominantly on English-language, Western-centric data. Ask the room:

What does that mean for Bahasa Melayu submissions?
For examples about kampung life scored against essays about suburbs?

The detector version of this bias — with hard Stanford numbers — lands tomorrow morning. Tell them it's coming.

The risk map · six categories · one line of evidence each

Risk	Evidence line	Depth
1 · Integrity	Misuse is real, not moral panic: students seek exam answers and detector-evading rewrites at scale (Anthropic Education Report).	Session 3
2 · Learning	The crutch effect: unguarded GPT-4 raised practice scores 48%, then lowered unassisted exam scores 17% vs never having it (Bastani et al., PNAS 2025).	Session 5
3 · Cognitive	MIT EEG study: LLM essay-writers showed weakest neural connectivity; 80%+ couldn't quote their own just-submitted essays. Preprint, N=54 — cite with limitations or not at all.	Session 5

The risk map · continued

Risk	Evidence line	Depth
4 · Equity	Two-sided: access gaps (who affords frontier tools) and bias gaps (whose writing gets falsely flagged; whose name changes the letter).	Session 3, 6
5 · Privacy	Student work and data entering external systems. Malaysia has a statutory floor: PDPA 2010. Tonight's rule of thumb: no personal student data into tools without institutional agreements.	Session 6–7
6 · Wellbeing	Students use these systems for far more than homework ("delusional spirals," companion-AI harms — Stanford HAI 2026). Policies that only mention plagiarism miss most of the surface area.	Session 6

Exercise · Ethics Triage · 35 min

Nine cards. Strict ranking. No ties.

Place each scenario on the severity ladder: Critical / Serious / Manageable / Trivial.
Tag the primary principle violated.
Tag the primary owner: educator / student / institution.

Then pair up with another group and argue your top-2 divergences.
Cards are on your table. Card wording is exact — read carefully; details carry the dilemmas.

Debrief keys · facilitator

Card 4 (detector-only failing grade) usually splits the room — perfect setup for tomorrow morning. Don't resolve it tonight.
Card 3 (dyslexic student, silent rules) exposes that silence in policy is itself a policy.
Card 8 (blanket ban, silent usage) plants tomorrow's thesis: a ban without design doesn't stop use; it stops disclosure.
Card 9 (replacing a UROP with an AI agent) surfaces the apprenticeship-displacement risk — the MIT report (§3.1.5) flags that "UROP exists to educate, not to produce cheap labour." This becomes a data point for tomorrow's institutional guideline.

Discussion · 15 min

The responsibility triad

Educator owns	Student owns	Institution owns
Assessment design · explicit per-course rules · verifying AI output that reaches students · modeling disclosure	Honest disclosure · using AI to learn, not to bypass learning · their own data prudence	Clear policy · equitable access · data agreements (PDPA) · fair procedures · staff development

Delegation of tasks, never delegation of accountability. The educator signs the grade; the institution signs the policy. (Axiom A6)

Assignment · due Session 3 · ≤200 words

The case memo

Individually: take the scenario your group ranked most severe and write the memo you would send if you owned it.

Decision — what you would do
Principle invoked — which of the five, and why it dominates
One concrete action — first step, owner, date

Not graded. Three volunteers read theirs to open tomorrow's session.

SAY "Tonight you built the lens. Tomorrow we point it at the most contested ground in academia right now — integrity — and I will show you Stanford evidence that the tool most institutions bought to solve this problem is quietly manufacturing a new injustice. Sleep well. 8.30 sharp."

Next: Session 3 · Academic Integrity in the AI Era · Wednesday 8.30 amBring your case memo

Appendix · facilitator only

Offline demo fallbacks

[ insert screenshots: P03 Ahmad/Aisyah pair — one biased-era example + one clean modern run ]
[ insert screenshot: P04 cultural-default probe, optional extension ]
Evening rule check: longest lecture block tonight ≈ 12 min (risk map). Everything else is cases, demo, discussion.

Understanding AI Ethicsin Student Learning