When ChatGPT was launched in November 2022, a number of debates emerged, many of which revolved around an age-old question: Which is better, human or machine?

Given the predicted importance that these tools will have in future professional practice, the real question is actually more nuanced: do humans do a better job on their own, or when they rely on artificial intelligence (AI)? More precisely: are they able to make good use of AI – to assess it correctly and correct its responses where necessary – to improve their performance?

A recent study suggests that this question is not as clear-cut as one might think. It concerns an assignment given in a class at HEC Paris (France's most prestigious business school), in which each student was randomly assigned two case studies.

For the first case, students had to write an answer to the assigned question from scratch. For the second, they received a ready-made answer, which they had to evaluate and, if necessary, correct. They were told that each answer could have been provided by ChatGPT, which was indeed the case for most of them. The students' final answers were graded using the same scoring system: the most important thing was to deliver a full reply to the question, whether it was the result of a correction or not ("Taking the help or going alone: ChatGPT and class assignments," June 2023).

While the first exercise is representative of "traditional" work practices, the second may end up corresponding more closely to many jobs in the future. Indeed, if AI tools become as ubiquitous as many predict, the human role will be to evaluate and correct the results produced by AI.

And yet, the students were less successful in this second exercise: the average mark for the corrected version of the ready-made answer was 28% lower than the average mark for the task in which students wrote their own answers. Even when controlling for the same case study, a student who corrected ChatGPT's work lost an average of 28 points out of 100 compared with a student working alone. In short, students perform much worse if they are provided with ChatGPT's help and then asked to correct it, than if they have to provide an answer from scratch.

Are these results due to high student trust in ChatGPT's answers? The students were explicitly warned to be wary of the answers they were given: they were told that ChatGPT had been tested on a similar assignment and that it had achieved rather mediocre results.

A more likely explanation would be confirmation bias, i.e. "interpreting evidence in ways that are partial to existing beliefs, expectations, or a hypothesis in hand."

You have 44.79% of this article left to read. The rest is for subscribers only.