Don’t rely on algorithms to make life-changing decisions

16th September, 2020

NEWS! My new book, “How To Make The World Add Up“, is out tomorrow around the world (except US / Canada). Ordering a copy early, online or from your local bookshop is enormously helpful: it prods review interest, encourages physical bookshops to order and display the book, and so I am especially grateful forearly orders. More information here – including a chance to buy signed copies.

The governments of England and Scotland have fed the hopes and dreams of students into a paper shredder, yanked out the tatters and handed them to university administrators with instructions to tape everything back together. The fiasco of algorithmically assigned exam grades is a nightmare for pupils, a huge embarrassment for those in charge and should be a cautionary tale for the rest of the world. With all too many classes cancelled in recent months, here, at least, is a teachable moment, with lessons that go far beyond education.

To summarise the train wreck: with schools closed and exams cancelled, but grades needed to assign places at university, pupils were promised that results would be forthcoming. The final grades were assigned by a data-driven view of each school’s historical record. No student, no matter how outstanding, would be awarded top marks if the algorithm concluded that her or his school was not a top-grade-sort-of-place. Countless individual injustices resulted.

To add insult, students sitting niche subjects in small classes were spared the harsh discipline of the algorithm. Because private schools benefited disproportionately from this selective indulgence it looked as though a government full of over-privileged dimwits was using an algorithm to favour the over-privileged dimwits of the future.

The U-turn part of the algoshambles was to wait until universities had reassigned their offers and only then cancel the downgrades in favour of the original school predictions, which were significantly higher. With many students suddenly entitled to university places that no longer existed, the entire rolling dumpster fire has now been pushed at university admissions offices.

The injustice and the incompetence here is palpable enough. But there is a deeper point about the intoxicating charm of algorithms. In March, when the government faced agonising choices about schools and exams, the algorithm promised fast, effective pain relief through the miracle of modern technology.

“When faced with a difficult question, we often answer an easier one instead, usually without noticing the substitution,” writes the psychologist Daniel Kahneman, in Thinking, Fast and Slow.

The difficult question here was: could we give students the grades they would have earned in the exams?

The easier substitute was: could we make the overall pattern of exam results this year look the same as usual? That’s not hard. An algorithm could mimic any historical pattern you like — or ensure equality (within the limits of arithmetic) based on gender or race.

But note the substitution of the easy question for hard. It is impossible to give students the right grades for exams they never sat — one would have to be infatuated with algorithmic miracles not to realise that.

But infatuation with algorithmic miracles is not new. One of the first computer dating services was called Operation Match. In the mid-1960s, it promised that the computer would “scientifically find the right date for you”. In fact, it mostly matched people who lived near each other.

An algorithm might try to predict romantic compatibility, but a wise couple would not marry on that basis without meeting. An algorithm might try to predict who will commit crime, and we might focus support on that basis. But I hope we will never dare to jail people for algorithmically predicted pre-crimes.

Neither should we make the life-changing decision to deny a university place for the pre-crime of presumptively missing a grade in a hypothetical exam.

Perhaps the least bad option this year was to focus on maximising access to jobs, apprenticeships and higher education, cramming more students into universities rather than playing the game of fantasy grades. Regardless, we would all be in a better position now if back in March the government had faced the truth rather than being dazzled by the sparkling promise of the algorithm.

Written for and first published in the Financial Times on 21 August 2020.

My NEW book How To Make The World Add Up is OUT TODAY! And of course it has a chapter on being sensible about the strengths and limits of algorithms…

Details, and to order signed copies from MathsGear, or from Hive, Blackwells, Amazon or Watersones.

Stephen Fry comments, “Fabulously readable, lucid, witty and authoritative.”

Pin It on Pinterest

Share This