ART-AI supervisor and lecturer Dr Tom Fincham Haines has recently been busy in the media commenting on the controversial A-Levels Algorithm as a result of his article A- levels: The Model is not the student.
Ofqual faced criticism over the design of its exam grading model, which led to many students losing out on University places. Tom studied the documentation released by Ofqual outlining how the algorithm was designed. He comments in this BBC article;
“Many mistakes were made at many different levels. This included technical mistakes where people implementing the concepts did not understand what the maths they had typed in meant”
The article highlights that as part of the process, Ofqual tested 11 different algorithms, tasking them with predicting the grades for the 2019 exams and comparing the predictions to the actual results to see which produced the most accurate estimates. However, the algorithms had already been given the 2019 results, meaning their predictive power was flawed from the outset.
According to Tom there was a need for far greater oversight of the process by which algorithms make decisions;
“A few hundred years ago, people put up a bridge and just hoped it worked. We don’t do that anymore, we check, we validate. The same must be true for algorithms. We are still back at that few hundred years ago stage and we need to realise that these algorithms are man-made artefacts, and if we don’t look for problems there will be consequences.”
The BBC article points out that algorithms are used at all levels of society, ranging from very basic ones to complex examples that utilise artificial intelligence;
“Most algorithms are entirely reasonable, straightforward and well-defined,” says Tom, but he warns that as they get more complex in design, society needs to pause to consider what it wants from them;
“How do we handle algorithms that are making decisions and don’t make the ones we assume they will? How do we protect against that?”
And, he states, some things should never be left to an algorithm to determine;
“No other country did what we did with exams. They either figured out how to run exams or had essays that they took averages for. Ultimately the point of exams is for students to determine their future and you can’t achieve that with an algorithm. Some problems just need a human being.”
Tom Haines was featured in a wide range of media including BBC Radio 5, the Financial Times , The Telegraph, and BBC News.
