Self-Driving Cars: Catch-22 Situation

Picture the scene: you're riding in a self-driving car when, after turning a corner, it faces an unavoidable collision with a group of 10 people in the road, with walls on either side. Should the car swerve into the wall — likely seriously injuring or killing you, its sole occupant — and save the group? Or should it try to stop, knowing it will hit the group while keeping you safe?

This is the moral and ethical dilemma tackled by a team of researchers led by Jean-François Bonnefon of the Toulouse School of Economics, in a paper published on arXiv in October 2015. They note that some accidents like this are inevitable as self-driving cars spread — and how the cars are programmed to react could play a huge role in public adoption of the technology.

"It is a formidable challenge to define the algorithms that will guide AVs [autonomous vehicles] confronted with such moral dilemmas," the researchers wrote. "We argue to achieve these objectives, manufacturers and regulators will need psychologists to apply the methods of experimental ethics to situations involving AVs and unavoidable harm."

In their study, the researchers surveyed several hundred people on Amazon's Mechanical Turk, an online crowdsourcing tool. They presented participants with scenarios like the one above, and varied the number of people in the car, the number of people in the group, and the ages of those involved.

The results are perhaps not too surprising: on the whole, people were willing to sacrifice the driver to save others — but most were only willing to do so when they did not imagine themselves as the driver. While 75% of respondents thought it would be moral to swerve, only 65% thought the cars would actually be programmed to swerve.

"On a scale from -50 (protect the driver at all costs) to +50 (maximize the number of lives saved), the average response was +24," the researchers wrote. "Results suggest that participants were generally comfortable with utilitarian AVs, programmed to minimize an accident's death toll."

The legal issues remain a grey area. Will new laws require the car to swerve — an emotionless "greater good" response? Or will cars be allowed to have different levels of morality? "If a manufacturer offers different versions of its moral algorithm, and a buyer knowingly chose one of them, is the buyer to blame for the harmful consequences of the algorithm's decisions?" the researchers ask.

Self-driving cars are inherently safer than human drivers — which creates a new dilemma of its own: if fewer people buy self-driving cars because they are programmed to sacrifice their owners, more people may die because ordinary cars are involved in so many more accidents. That is the Catch-22.

Self-driving cars are the logical future for public transport and promise to revolutionize travel around the globe. But as this study highlights, significant challenges still need to be addressed.

"Figuring out how to build ethical autonomous machines is one of the thorniest challenges in artificial intelligence today," the researchers wrote. "As we are about to endow millions of vehicles with autonomy, taking algorithmic morality seriously has never been more urgent."