Ethics study shows who we would save and kill
An ethical study intended to guide the behaviour of self-driving cars has found people are more likely to prioritise the lives of the young over the elderly, fit over unhealthy people, and humans over animals when faced with unavoidably tragic situations.
Researchers from universities, including the Massachusetts Institute of Technology, built “The Moral Machine”, a simple online survey which uses pictures to ask respondents to choose their preferred outcome in a wide variety of scenarios. The intention is the data could be used to guide ethical principles of autonomous vehicles in hopeless situations.
For example, some people were asked to decide whether it was better for a self-driving car to crash into a wall, killing three occupants, or to swerve onto the other side of the road, killing one pedestrian.
The research team says there is a need "to gauge social expectations about how autonomous vehicles should solve moral dilemmas”, particularly when distributing “harm they cannot eliminate”.
“Think of an autonomous vehicle that is about to crash, and cannot find a trajectory that would save everyone. Should it swerve onto one jaywalking teenager to spare its three elderly passengers?” they said.
“Even in the more common instances in which harm is not inevitable, but just possible, autonomous vehicles will need to decide how to divide up the risk of harm between the different stakeholders on the road.”
The Moral Machine received nearly 40 million decisions from millions of people in 233 countries and territories including Australia.
On average, people showed a preference for sparing humans over pets such as cats and dogs.
People elected to spare the lives of many over a few (for example, electing to kill one pedestrian to save three people in a vehicle), though there was a small preference in favour of saving pedestrians as opposed to motorists.
A greater number of people elected to spare women over men, favouring fit people as opposed to overweight people, along with young over the elderly, high-status individuals (such as doctors) over people with less social value (such as thieves) and law-abiding citizens.
Babies in strollers were the number one priority when choosing who to spare, followed by young children, pregnant women, doctors and athletes.
A noticeable number of people chose to sacrifice overweight, older or criminal members of society.
Intriguingly, more people elected to spare dogs than criminals, while cats were least likely to be spared.
Researchers found respondents in Eastern countries such as China or Japan were more likely to spare the elderly than those from Western nations or South America.
Women were generally spared at the expense of men, with the post-survey report finding men are “seen as more expendable” in developed nations with good health outcomes for women.
Countries grouped into a “Southern Cluster”, including Brazil, France and Argentina, were particularly predisposed to prioritising the lives of young, fit people. Wealthier countries were more likely to be biased against pedestrians who break the rules.
The team behind the survey admits they offer a relatively simple look at incredibly complex issues. There was no uncertainty in the dilemmas posed - respondents knew exactly what the fate of each character would be, and had a simple understanding of what they represented. Though millions of people responded to the survey, around 70 per cent of respondents were male, with the majority aged under 40. The most common income reported was less than $5000 per year, suggesting many students took part.
While there were broad trends across cultural and socio-economic divides, the research team found that “even the strongest preferences expressed through the Moral Machine showed substantial cultural variations”.
“Never in the history of humanity have we allowed a machine to autonomously decide who should live and who should die, in a fraction of a second, without real-time supervision. We are going to cross that bridge any time now, and it will not happen in a distant theatre of military operations; it will happen in that most mundane aspect of our lives, everyday transportation,” the report said.
“Before we allow our cars to make ethical decisions, we need to have a global conversation to express our preferences to the companies that will design moral algorithms, and to the policymakers that will regulate them.
“Whereas the ethical preferences of the public should not necessarily be the primary arbiter of ethical policy, the people’s willingness to buy autonomous vehicles and tolerate them on the roads will depend on the palatability of the ethical rules that are adopted.”
Car companies do not currently program cars to decide the fate of road users.
Drive rode shotgun with Mercedes-Benz safety experts during a research and development tour in Australia and South Africa in 2017.
When posed with the hypothetical “trolly problem”, an ethical question popularised by Netflix’s The Good Place which asks whether it is better to do nothing and kill several people in the path of a “trolley”, train or tram, or to divert the path of the vehicle and kill only one person, Mercedes spokesman Bernhard Weidemann says the question is largely theoretical, as autonomous cars are unlikely to travel at speeds potentially fatal to pedestrians without a strong awareness of their environment and enough room to stop safely.
“You wouldn’t go at a speed where you could not stop if something unexpected happened,” he said at the time.
“A self-driving car is extremely more unlikely to encounter the problem than a human. Why is that?
“If you all of a sudden have a pram or a trolley in your way it’s because you have not been paying attention, or you’ve been speeding around somewhere where you could not see the entire distance you could stop in, or you were playing with your phone.
“That’s where you come into a problem - you had a problem before you got into that.”
Mercedes safety expert Jochen Haab told Drive in 2017 that “the car at the moment will only brake.”
“It only offers swerving if I decide I want to swerve. We have no decision-making at the moment.”
To clarify, modern vehicles with autonomous emergency braking systems can detect obstacles such as pedestrians, and slam on the brakes to avoid a crash. High-end vehicles from the likes of Mercedes and Audi will only offer steering assistance to help avoid an obstacle if the driver starts to steer first - drivers still dictate a vehicle’s path.
Australian academics offered a variety of responses to the issue.
Hussein Dia, Associate Professor in Transport Engineering at the Swinburne University of Technology, says The Moral Machine “not only provides fascinating insights into the moral preferences and societal expectations that should guide autonomous vehicle behaviour, it also sets to establish how these preferences can contribute to developing global, socially acceptable principles for machine ethics”.
“These findings clearly demonstrate the challenges of developing globally uniform ethical principles for autonomous vehicles,” he says.
“Regardless of how rare these unavoidable accidents will be, these principles should not be dictated to us based on commercial interests, and we (the public) need to agree beforehand how they should be addressed and convey our preferences to the companies that will design moral algorithms, and to the policymakers who will regulate them.”
Distinguished Professor Mary-Anne Williams, Director of Disruptive Innovation at the Office of the Provost at the University of Technology Sydney, says there are many questions surrounding the ethics of autonomous cars, such as outside access to a vehicle’s decision-making logic.
“Who will have access to the data in autonomous vehicles ‘black box’?” she says.
“Will loved ones have the right to know all the autonomous car decisions. Will autonomous cars negotiate the outcome of multi-vehicle accidents? How will they resolve their inconsistent human-life preserving strategies during an accident? Without coordination, many more people may die unnecessarily.
“In order to minimise liability, car companies may design cars that slowdown in wealthy neighbours, or that kill humans rather than cause more expensive seriously injuries.”
Professor Lin Padgham at RMIT University pointed out that “the complex ethical/moral judgments required by some of the questions posed are not made by humans when confronted with these situations, and should not be expected of autonomous vehicles either”.
Professor Toby Walsh at the CSIRO and Commonwealth Bank’s Data61 research group agreed that people should treat the survey with “immense caution”.
“How people say they will behave is not necessarily how they will actually do so in the heat of a moment,” he says.
“I completed their survey and deliberately tried to see what happened if I killed as many people as possible. As far as I know, they didn't filter out my killer results.
“Also, whilst such studies tell us about people's attitudes it does not tell us how autonomous cars should drive. The values we give machines should not be some blurred average of a particular country or countries.
“In fact, we should hold machines to higher ethical standards than humans for many reasons: because we can, because this is the only way humans will trust them, because they have none of our human weaknesses, and because they will sense the world more precisely and respond more quickly than humans possibly can.”
Associate Professor Iain MacGill from the School of Electrical Engineering and Telecommunications at the University of New South Wales asked “can we trust the companies driving this, some with significant questions about their own ‘winner takes all’ business ethics, to appropriately program socially agreed ethics into their products?".
Full report: Nature
Try The Moral Machine here.
David McCowen is Drive’s news editor, combining automotive passion with more than a decade of reporting experience. Dave is often found at a racetrack – either in the press room, or driving his hot hatch.