CCW Catch-up: Consensus, But Too Many Concessions

This is Anna Hehir, FLI’s Head of Military AI Governance, Maggie Munro, Communications Strategist, and Éle Donegan, Military AI Fellow here with The Autonomous Weapons Newsletter. We’re excited to bring you the news on autonomous weapons systems (AWS) at a pivotal moment, as the world comes to terms with whether algorithms should make life and death decisions (spoiler alert: most people are terrified).

With this publication, we’re keeping our audience - primarily consisting of policymakers, journalists, and diplomats - up-to-date on the military AI space, covering policymaking efforts, weapons systems technology, and more.

That being said, if you have no idea what we’re talking about, check out autonomousweapons.org for our starter guide on the topic.

If you’ve enjoyed reading this, please be sure to subscribe and share as widely as possible.

CCW Group of Governmental Experts (GGE) Meeting on Lethal Autonomous Weapons Systems

We’re just back from Geneva where states, observers, civil society, and experts met at the UN for the final session of a three-year mandate to discuss autonomous weapons within the Convention on Certain Conventional Weapons. For a full readout on what was said between 31 August - 4 September, including analysis of each day’s discussions, do check out Reaching Critical Will’s report on this final GGE session. For now, here’s our take.

At 2:49am on Saturday morning, diplomats reached consensus on a text which could become the draft for a potential treaty on AWS. Admittedly, we wanted much more for the text. The most important principle - “meaningful human control” (R.I.P.) - has now been diluted to “human beings exercise such control directly or indirectly, including through measures taken before and/or during the use of AWS.”

If that doesn’t make sense to you, that’s because it doesn’t make sense. The most important concept to come out of the 12 years of UN discussions on autonomous weapons systems has been reduced to a sentence so nonsensical that not even ChatGPT could explain what it means.

The text did not include mention of ethical considerations, legal reviews of systems after modification, or references to the concepts of reliability, predictability, and traceability. It was essentially a recall of existing international law. Borrowing language used by diplomats this week, this text didn’t just fail to break through the ceiling, it was the floor.

*Collective sigh* Let’s directly and/or indirectly control ourselves before losing all motivation after this hit.

So why was this week important despite a weakened text?

Well, we have a text. And it is one that every state party to the CCW (128 High Contracting Parties, or HCP) has agreed to. This rare example of consensus means that we are one step closer to the world’s first international agreement covering issues related to AI. This achievement is easy to forget when we are wrapped up in the weeds of debating ‘determine’ from ‘determines’, ‘should’ from ‘must’, ‘human’ from ‘human being’. But when we do zoom out, it is important to remember that this discussion is history in the making. If this current text is as strong as a potential draft treaty text can get, then we have failed to produce new legal rules to protect humanity from the harms of autonomous weapons systems. So, let’s remember that we must - not just should - be on the right side of history and keep pushing for stronger prohibitions and regulations on AWS.

Through two joint statements presented by Algeria and Brazil on the opening day of discussions in Geneva, over 70 states (the majority of HCP to the CCW) have expressed that they are ready to launch negotiations. The moment and momentum has come.

Due to the mixture of celebration and frustration from this past week in Geneva, there is notable momentum from states and civil society to launch formal negotiations in November. The 2023 mandate is over and so the natural (and necessary) next step is progression from discussions to negotiations. If states progress and launch negotiations in November at the Review Conference of the CCW, it will do the following:

1. Answer the renewed call of the Secretary General of the United Nations and the President of the ICRC to launch negotiations urgently.

2. Signal to the international community that diplomats are taking note of the increasing evidence of harms caused by AWS and are doing something about it. This will serve as a reminder to the world that multilateralism works.

3. Reflect the will of the majority of High Contracting Parties (over 70 states) to the CCW as was stated in the joint statements delivered by Brazil and Algeria on the opening day of the Geneva discussions.

There are whispers of some militarized states suggesting a renewed GGE mandate of days, weeks or years, to further discuss the issue, keeping it frozen in bureaucratic perpetuity. This must not happen. It will cost lives. It will lead to the same outcome we saw this week: the principles that could save lives if adopted will be deleted in the name of ‘consensus’.

There is a lot of work to be done. The consequences for failure from states in November are high. In UN language speak, we are gravely concerned. But we’re also cautiously optimistic. Many governments around the world have a handful of dedicated and driven officials who care about this issue and have the power to push their systems to launch negotiations of an instrument. We’ll make sure to keep you updated along the way.

New updates to Autonomous Weapons Watch database

As different types of military AI are being further developed and used, we’ve updated our public database Autonomous Weapons Watch to capture global trends. Updates include:

• Additional systems in the autonomous weapons database

• A new database on decision support systems

• The publication of our first Trend Report

This wave of new content reflects the many technical trends we have been following in the world of Military AI. We’d love to hear your thoughts.

UN High Level Week

We’re getting ready to head to NYC later this month for the United Nations General Assembly High Level Week where world leaders will convene to take the stage, revitalize global cooperation, and face some serious traffic jams. An exciting event awaits at the Permanent Mission of Nigeria featuring FLI Chair Max Tegmark. We can’t wait to update you. Stay tuned as things take flight…

What we’re reading

• 📖 Military AI targeting algorithms can seem complex, but Airwars and AI Now Institute have broken it down for us in their interactive resource the Anatomy of a Kill Chain which shows the cumulative impact and devastation that occurs when AI is integrated into every stage of the military kill chain. If you click one link in this newsletter, let it be this one.

• 📖 This working paper by the ICRC which outlines their red lines for any future treaty on AWS.

• 📖 This working paper by Belgium, France, Germany and Singapore which discusses the different options for an instrument to address AWS, and the different avenues to negotiate such an instrument within the CCW.

• 📖 This piece from Magnus Løvold at Lex International, explaining the dynamics at play in the run up to the final GGE meeting.

• 📖 This opinion piece on AI-Enabled Targeting and the Structural Strain on International Humanitarian Law from Taylor Kate Woodcock, a researcher in public international law at the T.M.C. Asser Institute, and Jessica Dorsey, an Assistant Professor of International Law at Utrecht University School of Law.

Overheard since we last caught up

• In May, his Holiness Pope Leo called in his Encyclical for the international community to “disarm AI”.

• A renewed call to adopt rules on AWS: “Negotiations must begin now to urgently adopt a legally binding instrument to regulate autonomous weapon systems with clear prohibitions and restrictions. A failure to act quickly and decisively will cost lives.” - The Secretary General of the United Nations and the President of the International Committee of the Red Cross, a week before the Geneva meeting.

• “Absolute predictability is not a realistic technical standard for learning systems” ... “there's a debate about what accountability means" - some of the nonsensical arguments we heard this week from militarized states who are against prohibitions on unpredictable systems and accountability measures. Eyebrows were raised…

• "I join calls for the urgent prohibition of weapons that can take lives without human involvement” - Volker Türk, the UN Human Rights Chief, announced in a press conference just days after the conclusion of the Geneva talks.

AWS in the news

• In August, the New York Times reported that a drone “guided entirely by AI” killed three civilians. There have not been additional reports confirming these claims of full autonomy within this system, therefore we at FLI are hesitant to classify this as a fully autonomous weapon. Regardless, this reporting reflects the growing evidence of the harm caused by AWS.

• “Let us call [AWS] what they are: killer robots”. In July, the UNSG went out of his way at the Global Dialogue on AI to reiterate that autonomous weapons systems are his main concern and that we must not “wait for atrocity” to act on the issue.

• In June, the Report of the UN Secretary General on Children and Armed Conflict reported that the year of 2025 broke records for the number of children harmed in conflict, with child deaths up 34%. The report cited autonomy and artificial intelligence in weapon systems as contributing drivers of this harm.

Contact us

For tips or feedback, don’t hesitate to reach out to us at [email protected].