Countdown to the CCW Review Conference

This is Anna Hehir, FLI’s Head of Military AI Governance, Maggie Munro, Communications Strategist, Éle Donegan, Military AI Fellow, and Ann Deer, UN Engagement Lead here with The Autonomous Weapons Newsletter. We’re excited to bring you the news on autonomous weapons systems (AWS) at a pivotal moment, as the world comes to terms with whether algorithms should make life and death decisions (spoiler: most people are terrified).

With this publication, we’re keeping our audience - primarily consisting of policymakers, journalists, and diplomats - up-to-date on the military AI space, covering policymaking efforts, weapons systems technology, and more.

That being said, if you have no idea what we’re talking about, check out autonomousweapons.org for our starter guide on the topic.

If you’ve enjoyed reading this, please be sure to subscribe and share as widely as possible.

An Ambitious Mandate: One Protocol in One Year

In under two weeks, States Parties to the Convention on Certain Conventional Weapons (CCW) will meet in Geneva to decide the next steps in the international governance of autonomous weapons.

In other words, it’s crunch time.

This meeting will be at the Seventh Review Conference (RevCon) of the CCW, where states will be discussing the five existing protocols of the CCW:

1. Non-Detectable Fragments

2. Mines, Booby-Traps, and Other Devices

3. Incendiary Weapons

4. Blinding Lasers

5. Explosive Remnants of War

There will be dedicated discussions on the topic of autonomous weapons, with many watching to see if they will conclude with agreement to launch formal negotiations on a sixth protocol to the CCW. This is what has been proposed by Peru, the Chair of the upcoming Geneva discussions.

These discussions are an opportunity to take what was agreed by consensus in September and launch negotiations that turn the set of elements into a new Protocol VI, a legally binding instrument.

Last week, His Holiness Pope Leo XIV implored states to do just that. During a dialogue between the Vatican and the Scientific World on the Developments and Applications of Intelligent Technologies, His Holiness encouraged “States meeting in Geneva in November to take concrete steps toward banning lethal autonomous weapons.”

As states prepare for the CCW Review Conference, diplomats have also been working hard in New York. During UN First Committee discussions, states from all corners of the world used their national statements to call for an ambitious mandate to be agreed at the Review Conference.

But what does ambitious mean? Well, here is how some states defined it:

Indonesia, speaking on behalf of the Non-Aligned Movement, called for an ambitious negotiating mandate to negotiate an appropriate legally binding instrument on LAWS.

Algeria, Bangladesh, Belgium, Brazil, Cuba, Egypt, Guatemala, the Holy See, Iraq, Ireland, Jamaica (on behalf of the 15 Caribbean Community states), Mexico, New Zealand, Nigeria, Pakistan, Senegal, Sri Lanka, and Zimbabwe also called for negotiations on a legally binding instrument.

At the CCW RevCon, states need to hold the line. There will be considerable pressure on many delegations. We will likely see some states frame a mandate to negotiate a non-legally binding instrument as a success. Multi-year negotiations on something that is not explicitly legally binding will not lead to credible regulation of autonomous weapons.

States should accept nothing less than an ambitious mandate to negotiate a protocol on autonomous weapons by the end of 2027.

It has been 23 years since the CCW agreed on an additional protocol. We cannot wait any longer. Many are watching to see if the forum can still produce new protocols to address new weapons.

If you’re reading this and thinking this sounds pretty important 🤔 - I want to help! But feeling a little unsure on what to do….👇

Poetry Interlude

In our very first edition we bestowed you with a haiku. There's something about the falling leaves and high pressure deadlines set to create new international law that has us counting syllables. There are alternatives to screaming into the void, and woefully bad poetry about niche topics is worth considering.

To block, to defend

Human oversight is gone

How many more years?

Do you have an equally bad haiku about multilateral diplomacy or the state of the world that you need to get off your chest? Are you sick of writing cables, reports and drafted WhatsApp messages? Send us an anonymous (or not) poem, and we may or may not create an anthology for those who also channel their rage into embarrassing creative writing projects.

UN First Committee Statement Shout-outs

During UN First Committee, 163 states voted in favor of the Austrian Draft Resolution L.37 calling for the launch of negotiations on an instrument on lethal autonomous weapons systems (LAWS). States overwhelmingly voted in favor, despite explicit pressure from the Russian Federation asking states to vote no, and claims from the United States that this resolution would threaten the CCW.

Below are some standout statements we heard in the run-up to the vote which ignored these nonsensical arguments and threats.

• Austria 🇦🇹 stated “We need international rules for autonomous weapons built around one clear principle. No machine may decide to take a human life.”

• Algeria 🇩🇿 emphasized that a legally binding instrument is “urgent and can no longer be deferred.”

• Bangladesh 🇧🇩 stated “We believe that the international community must take concrete steps to ensure meaningful human control over the use of force.”

• Brazil 🇧🇷 stated “The rapid integration of artificial intelligence into military capabilities demands urgent multilateral action… The upcoming CCW Review Conference must advance concrete negotiations of a protocol based on the GGE report.”

• Costa Rica 🇨🇷 stated “It is urgent to ensure that decisions regarding the use of force remain at all times under meaningful human control.”

• Egypt 🇪🇬 declared “There is a need to start, and without delay, negotiations on a legally binding instrument to govern the production and use of lethal autonomous weapon systems.”

• The Holy See 🇻🇦 reiterated that the ”decision to use lethal force cannot be delegated to opaque or automated processes, but must remain under effective, self-aware, and responsible human control.”

• Indonesia 🇮🇩 called “for an ambitious negotiating mandate building upon the substantial work already undertaken with a view to negotiating an appropriate legally binding instrument on LAWS.”

• Ireland 🇮🇪 stated that “we must now move to negotiations on that instrument, which we believe must be legally binding, to prohibit systems that are incompatible with international humanitarian law and to regulate others.”

• Jamaica 🇯🇲, on behalf of the Caribbean Community states, stressed “that the development of such weapons and their potential use raise legal, ethical, and security concerns. These systems must always remain in strict compliance with international law and under human control and oversight.”

• Nigeria 🇳🇬 stated “human judgment, control and accountability must remain central to decisions involving the use of force.”

• Norway 🇳🇴 declared “LAWS are no longer science fiction. They are here.”

• Sri Lanka 🇱🇰 said it was “deeply troubled by the prospect of lethal autonomous weapon systems. We support the early commencement of negotiations on a legally binding instrument to prohibit LAWS, and became one of the main sponsors of the First Committee resolution on LAWS.”

The Numbers Are In: United Kingdom, Germany, and France

New polling commissioned by the Future of Life Institute, conducted by Dynata, in the United Kingdom, Germany, and France found that around 9 in 10 respondents across the three countries support an international treaty on AI weapons.

Let’s break it down:

94% in the UK.

87% in Germany.

92% in France.

Across the three countries, 88% of respondents were concerned about the rising use of AI weapons in war and over 85% in all countries were in favor of their government leading on efforts to regulate AI weapons, specifically 93% in the UK, 86% in Germany, and 89% in France.

The people of the United Kingdom, Germany, and France have spoken. Not only do they want a treaty on AI weapons, but they want their countries to lead it.

For more information on this polling, check out our coverage here.

Welcome to Tech Bytes

Ahead of our upcoming Trend Report, we wanted to remind readers of the findings of our August edition by incorporating a new segment to the newsletter where we offer a technical explainer on a military AI topic.

This edition, we are talking about swarming.

Swarming is the coordinated operation of multiple unmanned systems that share information and allocate tasks among themselves, rather than each being controlled individually. We have not seen evidence that operational swarming has been achieved by any actor to date. Most systems marketed as “fully autonomous” swarms are closer to coordinated multi-drone control, mesh networking, or autonomous teaming.

The Ukrainian start-up Swarmer AI has made a number of statements regarding its development of swarming capabilities, including that one “swarm” can include up to 690 drones.

Sources on the ground believe that it is unlikely that the case of one swarm of “690 drones” represents the technical achievement of fully autonomous swarming. They report that a mass of unmanned aerial systems (UASs) interacting and controlling themselves without human input is not yet feasible. They believe this mass deployment of UASs exhibits some autonomous features, such as navigation. The interactions of individual systems remain controlled by a single command center, rather than by systems autonomously coordinating with each other. This command center may be staffed by one or more humans.

Have We Reached Consensus on Consensus?

We want to take this opportunity to do a deep dive on a paper featured in the Reaching Critical Will’s First Committee Monitor by Yvette Issar of the Quaker United Nations Office: “All the ways it hurts: Compounding disadvantages of the CCW’s consensus practice for constructive States.”

In this piece, Issar discusses how the practice of decision-making by consensus, despite seeking to produce outcomes that are acceptable to all participants, has led to compounding negative impacts for international governance. Some states use and abuse consensus, wielding it like a veto power, while most states play by a different set of rules.

The result? A dysfunctional and asymmetrical playing field.

This is a worthwhile read for anyone with an interest in multilateralism and bodies that are hijacked by consensus voting. Rather than summarize its many excellent points, we will leave you with Issar’s closing sentiment.

“In the CCW, consensus has gradually shifted from an aspiration to a requirement without which action is impossible. Such a consensus-only system is vulnerable to strategic exploitation, since any actor’s ability to withhold agreement can generate disproportionate leverage. At the same time, questioning the current practice of consensus within the forum may be construed as disloyalty to the institution or to multilateralism more broadly. This discourages critical examination of the practice and consideration of alternatives. But critical questioning is not only compatible with a commitment to multilateralism, it is necessary to preserve the agency of all participants in the system.”

What we’re reading and watching

• Dr. Emily Camins of the University of Western Australia Law School wrote a powerful piece about the importance of supporting the victims of AI weapons in our governance efforts.

• 🍿 Lex International released their documentary Will We Let AI Kill People? on YouTube. You’ll spot some familiar FLI faces in there too. 👀

• Another incredibly important documentary: NAZA. This film from Yuval Abraham and Rachel Szor, the Israeli Oscar-winning No Other Land filmmakers, received a 25-minute standing ovation at the Venice Film Festival. It features 24 anonymous testimonies of individuals familiar with the integration of AI into Israeli military action in Gaza. In late November, it will be released on the Guardian’s website for free.

• Military AI continues to be all over social media. We’ve been following ctrl.arms for some snappy short-form content to share with family and friends. Alžběta, we love seeing you on our feed.📱

Overheard this month

• “Over three years, a small number of militarily significant states, the very states developing and fielding these technologies, worked steadily to thin it [Rolling Text on LAWS] out,” wrote Husham Ahmed, a Pakistani diplomat, as he reflected on the trajectory of the LAWS discussions.

• “AI weapon systems are already here” wrote former Human Rights Watch Executive Director Kenneth Roth in a Guardian op-ed.

• “No safety testing, voluntary commitment or corporate policy substitutes for international rules where a new technology of warfare creates humanitarian risks that the world cannot accept. That is a decision for states, made together, in public, and one they can be held to” wrote Dr. Cordula Droege, Director of International Law, Movement and Partnerships at the ICRC, commenting on the importance of legally binding international governance in her blog post.

• “Drones are able to seek out and eradicate targets without any sort of human intervention” explained Sophia Goodfriend, a Journalist and Research Fellow at the University of Cambridge in Move Fast, Break People, a short film by Stop Killer Robots.

Military AI in the News

• Former Google employee Andreas Kirsch believes AI technology in warfare is not appropriate for the battlefield. "Current AI models are not robust enough, and I don't think they have been trained for such tasks either." We implore everyone to read this testimony from tech workers resigning from Google DeepMind over the company’s military AI contracts.

• “Utterly chilling, utterly terrifying” said Tom Fletcher, United Nations Under-Secretary-General for Humanitarian Affairs and Emergency Relief Coordinator in an interview with RTÉ. His commentary was in reaction to his recent time in Somalia, Sudan, South Sudan, and Ukraine where he witnessed the “lethality of drones increasingly powered by AI.”

• The Houthis, a non-state armed group, tried to use Anthropic’s Claude AI chatbots to build ballistic missiles.

• The United States Department of War has announced the start of Project Agincourt, which will create the Autonomous Warfare Command (AutoWarCom) to scale autonomous and robotic capabilities across the military.

Contact us

For tips or feedback, don’t hesitate to reach out to us at [email protected].