In the late 1990s, the UK Post Office deployed a new point-of-sale computer system, built for it by Fujitsu. Almost immediately, post-masters, who are self-employed and generally small independent retailers, started reporting that it was showing shortfalls of cash; the Post Office reacted by launching prosecutions for theft. Close to a thousand people were convicted over the next 15 years, many more were falsely accused and forced to settle, and there were at least four suicides.

The system was full of bugs that could cause false shortfalls to appear, and some people inside Fujitsu and the Post Office knew that, but Fujitsu and Post Office staff went to court and testified that the system was working correctly and that theft was the only explanation. This has now, understandably, become a huge scandal.

I think about this case every time I hear about AI Ethics and every time people talk about regulating AI. Fujitsu was not building machine learning or LLMs - this was 1970s technology. But we don’t look at this scandal and say that we need Database Ethics, or that the solution is a SQL Regulator. This was an institutional failure inside Fujitsu and inside the Post Office, and in a court system failing to test the evidence properly. And, to be clear, the failure was not that there were bugs, but in refusing to acknowledge the bugs. Either way, to take the language that people now use to worry about AI: a computer, running indeterminate software that was hard to diagnose or understand, made ‘decisions’ that ruined people’s lives - it ‘decided’ that money was missing. The staff at the Post Office just went along with those decisions.

We don’t solve this problem with a SQL regulator, and the same point applies when we read that FTX had a spreadsheet with eight different balance sheets. We don’t call for a spreadsheet regulator, and demand that Microsoft stop this from happening again. That would be the wrong level of abstraction.

This, I think, is the challenging in talking about ‘AI Ethics’, or in writing laws to regulate ‘AI’.

There are lots of ways that bad people can do bad things with software, and there are lots of ways that people can screw up with software - we worry what happens if the technology works and we worry what happens if it doesn’t work. In the last decade, machine learning created a bunch of new ways to screw up or be evil, and generative machine learning will do the same.

However, those problems will occur in very different forms in very different places. People will use AI for everything from parole processing to granting mortgages, from marking middle-school tests to spotting thieves (or trying to) in shopping malls, to optimising wind turbine blades and laying out cycle lanes. Some of those have ethics issues, or bias issues (and some fo the bias issues will even be bias about people). Some of them have scope to screw up and kill people. But they’re all different problems, with different questions and consequences and different kinds of expertise.

The fuss in the last few weeks about Google Gemini is a nice case-study of this. Most people, I think, agree that if you ask a search engine for the best ways to kill yourself, it shouldn’t just tell you the trade-offs between a noose and sleeping pills, and we also understand that Instagram and Pinterest probably shouldn’t promote self-harm content to 14 year old girls, even if that’s what they seem be interested in. But we’ve spent a decade or more arguing about what content moderation really means and what kinds of lines you draw, and now we need to apply those arguments to generative search or generative images, and that won’t be any easier. Lots of clever people will enjoy spending the next few years arguing about this - but meanwhile, that has nothing do with training data and testing protocols for generative AI in drug discovery for multiple sclerosis. These are different problems, and it is very hard to see a field of ethics that covers all of then. As Larry Tesler said, AI is whatever doesn’t work yet: once it works it’s just software, and everything is software now.

I often talk about tech regulation in comparison to regulating cars. Cars cause all sorts of issues, and we have lots of rules and policies, and our current wave of regulation around consumer technology looks pretty similar sometimes. But we don’t have one government department, and one omnibus law, to cover how GM treats its dealers, collision and safety standards, congestion charging in big cities, whether the tax code encourages low-density development, what to do about teenaged boys drinking and driving too fast, and the security of national oil supplies. These are all big questions, and generative AI raises all sorts of possible problems too, but they’re all different things, best understood by very different people.

The big difference between tech and cars is that we took 75 years to put seatbelts in cars and we’re not waiting 75 years to regulate tech, and yet we all grew up with cars and understood them, where we often don’t intuitively understand the issues with some technology problems. Generative AI adds another layer to this: Instagram or Tiktok might be pretty new, but they look much the same now as they did last year, whereas no-one in tech really knows what generative AI will look like at the end of this year. There are all sorts of really basic questions about how this works and how it will progress that are wide open. To stretch my analogy, we’re writing laws and theses about aeroplanes and motor-cars in 1910. That requires some humility, and an expectation that most of what you say might be irrelevant this time next year.

We Are All Bewildered Machines - Issue 66: Clockwork