Existential Risk Observatory News
Open lecture and debate Dec 11th @ Lab111, Article on a Conditional AI Safety Treaty in TIME, and much more
“May we expect aligned superintelligence?” - Open lecture and debate with Simon Friederich (RUG)
Artificial intelligence could change our world beyond recognition. When AI surpasses human level, we could face an existential threat to humanity. AI alignment, or making sure AI acts in accordance with our values, is often seen as a solution.
But is it even possibe to program those preferences of AI so that they align with those of humans? And which human values and preferences are we speaking of anyway?
Be sure to register through this page!
Date: this coming Wednesday December 11th
Time: 20.00 - 21.30, followed by drinks
Location: Lab111, Arie Biemondtstraat 111, Amsterdam
Article on a Conditional AI Safety Treaty in TIME
Existential Risk Observatory proposes the Conditional AI Safety Treaty in TIME as a solution to AI's existential risks.
AI poses a risk of human extinction, but this problem is not unsolvable. The Conditional AI Safety Treaty is a global response to avoid losing control over AI.
How does it work?
AI Alignment has so far been presented as a solution to existential risk. However, alignment has three main problems:
- It is scientifically unsolved.
- It is unclear which values we should align to.
- Having one friendly AI does not necessarily stop other unfriendly ones.
Therefore, building upon the "if-then commitments" proposed by Bengio, Hinton, and others in "Managing extreme AI risks amid rapid progress" we propose a treaty where signatories agree that IF we get too close to loss of control AND alignment is not conclusively solved THEN they will halt unsafe training within their borders.
This treaty solves two issues:
1) Coordination. It is in the interest of signatories to verify each others’ compliance, and to make sure dangerous AI is not built elsewhere, either.
2) Timing. Some say AGI is nowhere near. We take their POV into account with the if-then part.
How close is too close to loss of control? This will remain a difficult question, but someone will need to answer it. We propose the AISIs do so. They have eval know-how, which can be extended to loss of control. Also, they are public and independent from the AI labs.
Under the Conditional AI Safety Treaty, we can still get most of AI's benefits. All current AI: unaffected. Future narrow AIs (climate modelling, new medicines, nuclear fusion): unaffected. Future general AIs safer than a threshold: unaffected.
The Trump government might bring opportunities. Ivanka Trump is aware of the urgency of the problem. Elon Musk is a known xrisker. Tucker Carlson is concerned, as is Trump himself. A Trump unified government could be able to get this treaty accepted by China.
We think our proposal is going in the same direction as many others, such as by Max Tegmark (Future of Life Institute (FLI)), Connor Leahy (Conjecture), and Andrea Miotti (ControlAI). We welcome their great work and are open to converging towards the most optimal solution.
We realize that a lot of work needs to be done to get the Conditional AI Safety Treaty implemented and enforced. But we believe that it we really want to, these challenges are by no means beyond humanity's reach.
We can solve xrisk, if we want to.
Read the full piece here
Save the date! Special Paris event featuring a renowned speaker on AI safety, February 9th, 2025
The AI Action Summit will take place in Paris this February. Heads of State and Government, leaders of international organizations, CEOs of small and large companies, representatives of academia, non-governmental organizations, artists and members of civil society will discuss the future of AI. We are happy that this is an inclusive event compared to the previous AI Safety Summits, but much less happy that existential risk is less prominently part of the programme. This is why we consider it all the more necessary that the existential risk AI poses will be in the spotlight ahead of the Summit.
Therefore, we will organize our own event on the evening of 9 February, where a globally leading AI voice will inform an audience of policymakers and leaders of the societal debate about the importance and urgency of AI's existential risks. We will also discuss policy options, among which our own Conditional AI Safety Treaty proposal.
Entrance will be free and open to all, as always. An online streaming link will likely be provided ahead of the event for those who cannot otherwise make it. Watch our social media channels to be kept up to date!
Other
News - US Commerce Secretary Gina Raimondo addressed the opportunities and risks of AI — including "the possibility of human extinction."
News - OpenAI and others seek new path to smarter AI as current methods hit limitations. It is now public knowledge that multiple LLMs significantly larger than GPT-4 have been trained, but they have not performed much better. That means scaling laws have broken down. What does this mean for existential risk? ERO’s Otto Barten produced this graph, with an update on the scaling law:
Donations - Have you considered donating to the Existential Risk Observatory this Giving Season? Any gifts would be much appreciated! Existential risk awareness building is funding-constrained. With additional funding, we could operate in more countries, organize more and better events, and do more research investigating the effects of our interventions. We are sincerely happy with all support, both large and small! You can either directly contact us or donate through this link.
