In February 2018, Regina Surber released a commentary ‘Autonome Technologie ist nicht nur in Kriegsrobotern riskant’ (eng.: Autonomous technology: the danger beyond killer robots), which was published in the Neue Zürcher Zeitung (NZZ) on February 21, 2018. The article can be found here. And the english version here and below.
The key message of the article is that it is high time to discuss peace-time threats of autonomous technology, an aspect of emerging technologies’ peace and security implications that is currently hardly addressed both by the public and the high-level UN discussions.
Download PDF of article here.
Autonomous technology: the danger beyond killer robots
The recent international and national media furore concerning the application of Artifical Intelligence in autonomous weapons systems and killer robots, also known as ‘LAWS’ (Lethal Autonomous Weapons Systems), and the short film ‘Slaughterbots’, have heightened our collective anxiety about the future of war and the nature of combat.
This intense media attention is overdue and justified as weapons systems that can identify, search, select and attack without human involvement in the robot’s ultimate decision-making processes already exist today and, with increasingly advanced algorithms, will be created in progressively sophisticated versions. The development of LAWS forces us to ask ourselves existential questions about the current state of the world, and to seriously examine present and future technological developments.
Some of the questions that arise are ethical in nature: should a person even be allowed to develop and then use an algorithm that can directly lead to the death of another human being? On the other hand, there are also fundamental questions about human identity itself and the concept of self: should the achievements of humans and machines be measured using the same metrics? Should we even compare humans and machines? Given the potentially critical impact of LAWS for humanity, a wider public discussion is needed that goes beyond the current conversations that are taking place at the highest levels of the UN concerning LAWS and the rules of armed conflict. The public debate about LAWS runs the risk of being quickly silenced ahead of time, because the controversy could seem to be well taken care of within the current framework of high-level discussions at the UN in Geneva.
Hence, one may ask why, for example, the Swiss public should be worried about killer robots? Firstly, the discussions taking place at the UN are only focused on the use of LAWS in times of war. However, autonomous weapons systems can also be used during national police operations, hostage situations and crowd control, and are currently being developed exactly for this kind of scenario by companies such as Desert Wolf. The UN debate does not address this application of autonomous technology nor the strategically lucrative uses of autonomous technology in cyber operations, for example NSA’s program known as ‘MonsterMind.’ Moreover, it also does not look at the possible risks of other new technologies such as 5G and biotechnology and their possible linkages with autonomous technology.
Secondly, autonomous technologies are not only dangerous when they are deliberately developed and used in weapons systems. They can also generate fake news and be implicated in public deception on a large scale. In addition, autonomous technology can be used to develop criminal profiles of offenders, which combined with its ability to independently assess and draw conclusions based on big data, blur the boundary between criminals and legally innocent people. CCTV cameras in Moscow and China are already being increasingly equipped with facial recognition software in order to continuously generate personal data of the faces and behaviour of the population.
We also need to think about whether or not we will be forced to keep the world’s population within artificial borders given that the current global financial and capitalist system does not distribute resources in a way that is satisfactory for large swaths of humanity. In this light, one could also imagine that autonomous software integrated into the health system could superficially remove morally difficult decisions concerning life and death of ‘superfluous’ human beings. Coupled with a rating system for citizens – China is introducing a prototype nation-wide called Citizen Score in 2020 – autonomous technology could be used for population growth control based on utilitarian calculations.
These thoughts force us to question our contemporary financial and economic systems, as well as our societal values. Murphy’s Law states that everything that can go wrong, eventually will. We therefore have a moral responsibility to minimize, as much as possible, the risks of these dangerous scenarios. Consequently, an overarching ethical debate is urgently needed.
LAWS should not be understood solely as a problem relating to killer robots, which the UN is trying to keep in check. We need to understand LAWS as a harbinger of a world in which human beings won’t be the only “intelligent system” with the ability for autonomous action.
The active involvement of civil society, academia, business, government and international organizations is needed to address these fundamental questions. Switzerland, in particular the city of Zurich, home to big IT and entertainment companies such as Google, IBM and Disney, and the world-renowned university ETH, is perfectly positioned to lead and shape future discussions and ideas on autonomous technology.