Airlines take no chances with our safety. And neither should artificial intelligence

You’d thinking flying in a plane would be more dangerous than driving a car. In reality it’s much safer, partly because the aviation industry is heavily regulated.

Airlines must stick to strict standards for safety, testing, training, policies and procedures, auditing and oversight. And when things do go wrong, we investigate and attempt to rectify the issue to improve safety in the future.

It’s not just airlines, either. Other industries where things can go very badly wrong, such as pharmaceuticals and medical devices, are also heavily regulated.

Artificial intelligence is a relatively new industry, but it’s growing fast and has great capacity to do harm. Like aviation and pharmaceuticals, it needs to be regulated.

AI can do great harm

A wide range of technologies and applications that fit under the rubric of “artificial intelligence” have begun to play a significant role in our lives and social institutions. But they can be used in ways that are harmful, which we are already starting to see.

In the “robodebt” affair, for example, the Australian government welfare agency Centrelink used data-matching and automated decision-making to issue (often incorrect) debt notices to welfare recipients. What’s more, the burden of proof was reversed: individuals were required to prove they did not owe the claimed debt.

The New South Wales government has also started using AI to spot drivers with mobile phones. This involves expanded public surveillance via mobile phone detection cameras that use AI to automatically detect a rectangular object in the driver’s hands and classify it as a phone.

Facial recognition is another AI application under intense scrutiny around the world. This is due to its potential to undermine human rights: it can be used for widespread surveillance and suppression of public protest, and programmed bias can lead to inaccuracy and racial discrimination. Some have even called for a moratorium or outright ban because it is so dangerous.

In several countries, including Australia, AI is being used to predict how likely a person is to commit a crime. Such predictive methods have been shown to impact Indigenous youth disproportionately and lead to oppressive policing practices.

AI that assists train drivers is also coming into use, and in future we can expect to see self-driving cars and other autonomous vehicles on our roads. Lives will depend on this software.

The European approach

Once we’ve decided that AI needs to be regulated, there is still the question of how to do it. Authorities in the European Union have recently made a set of proposals for how to regulate AI.

The first step, they argue, is to assess the risks AI poses in different sectors such as transport, healthcare, and government applications such as migration, criminal justice and social security. They also look at AI applications that pose a risk of death or injury, or have an impact on human rights such as the rights to privacy, equality, liberty and security, freedom of movement and assembly, social security and standard of living, and the presumption of innocence.

The greater the risk an AI application was deemed to pose, the more regulation it would face. The regulations would cover everything from the data used to train the AI and how records are kept, to how transparent the creators and operators of the system must be, testing for robustness and accuracy, and requirements for human oversight. This would include certification and assurances that the use of AI systems is safe, and does not lead to discriminatory or dangerous outcomes.

While the EU’s approach has strong points, even apparently “low-risk” AI applications can do real harm. For example, recommendation algorithms in search engines are discriminatory too. The EU proposal has also been criticised for seeking to regulate facial recognition technology rather than banning it outright.

The EU has led the world on data protection regulation. If the same happens with AI, these proposals are likely to serve as a model for other countries and apply to anyone doing business with the EU or even EU citizens.

What’s happening in Australia?

In Australia there are some applicable laws and regulations, but there are numerous gaps, and they are not always enforced. The situation is made more difficult by the lack of human rights protections at the federal level.

One prominent attempt at drawing up some rules for AI came last year from Data61, the data and digital arm of CSIRO. They developed an AI ethics framework built around eight ethical principles for AI.

These ethical principles aren’t entirely irrelevant (number two is “do no harm”, for example), but they are unenforceable and therefore largely meaningless. Ethics frameworks like this one for AI have been criticised as “ethics washing”, and a ploy for industry to avoid hard law and regulation.

Another attempt is the Human Rights and Technology project of the Australian Human Rights Commission. It aims to protect and promote human rights in the face of new technology.

We are likely to see some changes following the Australian Competition and Consumer Commission’s recent inquiry into digital platforms. And a long overdue review of the Privacy Act 1988 (Cth) is slated for later this year.

These initiatives will hopefully strengthen Australian protections in the digital age, but there is still much work to be done. Stronger human rights protections would be an important step in this direction, to provide a foundation for regulation.

Before AI is adopted even more widely, we need to understand its impacts and put protections in place. To realise the potential benefits of AI, we must ensure that it is governed appropriately. Otherwise, we risk paying a heavy price as individuals and as a society.