Paul Patterson was repentant. Fujitsu was “truly sorry”, the head of its European business told MPs on January 16th, for its role in Horizon, a faulty payments system that resulted in the false conviction of over 900 Post Office sub-postmasters between 1999 and 2015. Why did the Japanese firm do nothing about bugs it knew bedevilled its software, even as tales of injustice, destitution and worse mounted? “I don’t know. I wish I knew but I just don’t know.”

Mr Patterson is the first Fujitsu executive to be questioned publicly about the scandal; others will be probed more forensically at a separate public inquiry. Yet already the evidence points to classic problems with the way public bodies contract firms to build and manage large IT systems—as well as to the risks of believing that computers are always right.

At the turn of the millennium many functions of the British state were still administered with pen and paper. The prime minister did not have an email account until 2003. Modernisation was a vast, urgent and difficult task, and one for which the civil service was wholly unprepared. There was a “strategic failure” to take IT skills seriously, says William Perrin, a technology adviser to Tony Blair in the early 2000s. The decades since have been littered with costly disasters.

Horizon is an egregious example, but also an instructive one. The most obvious problem was that the Post Office, which is a public corporation, was a credulous customer. Senior executives had no idea how branches worked in practice or what a “computerised” payments system would involve, according to John Murray, a former Post Office project manager.

In the tender for the contract, the tactic of ICL, a British firm that developed Horizon and was fully absorbed into Fujitsu in 2002, was to bid low. IBM and Cardlink produced better pitches that were focused on “difficulties and complexities”, says Mr Murray. Executives plumped for cheap promises. Outsourcing disasters have often come from departments or agencies blindly selecting lowball bids, reckons the Institute for Government, a think-tank.

Contract oversight was poor. Paula Vennells, the CEO of the Post Office between 2012 and 2019, responded to concerns about Horizon by referring to assurances from Fujitsu that the software was “like Fort Knox”. What that really meant was the Post Office did not know what was going wrong. Jeremy Folkes, the employee tasked with ensuring the system worked, called it a “black box”; the Post Office had no access to design documentation or to code.

Another procurement problem is that the penalties for poor performance are light. Fujitsu has had a patchy reputation in Whitehall yet ministerial efforts to block it from winning new contracts largely failed. The threat of legal challenge under EU competition rules, and a scarcity of big suppliers, have allowed it to keep landing work (see chart). Proprietary code and the need to keep services running make failing IT contracts hard to escape from.

The IT cowed

The obvious solution to such problems is more digitally capable government. Between 2010 and 2015 the Government Digital Service (GDS), a team of tech-savvy upstarts within the civil service, notched up some successes, including to how contracts were run. But GDS’s influence waned; the best techies went back to the private sector; skills remain patchy.

The Post Office scandal also invites a bigger question. Even as sub-postmasters raised concerns, it was easy for the Post Office and Fujitsu to insist that the system was robust. That is partly because of a law, which MPs are now seeking to change, that assumes that computer evidence is reliable. But it also reflects a mindset that is in evidence well beyond Britain. In the Netherlands an automated system for predicting child-benefit fraud malfunctioned; more than 1,000 children were taken into care before the prime minister, Mark Rutte, was forced to resign. In Australia Robodebt, an algorithm for recovering debts from benefit claimants, similarly went badly wrong. The government eventually had to pay victims A$1.8bn ($1.2bn).

The potential for technology to improve the public sector is vast. But those in charge of government institutions cannot blindly rely on computers. When humans say there is a problem, they must listen. ■

For more expert analysis of the biggest stories in Britain, sign up to Blighty, our weekly subscriber-only newsletter.