. |`. _..--''''--.._ . | `. .-'' | | | | ''-. |`. | \ . _.'| | | | | | | |'._ . | `| \ |`. |=|==========================|=| |`. | | | . | \ | |/ \| | | `| | | |`. | | | | | | __|___|___|______|_____|__`|___|__ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |________________________________| ~~~ ~~~~~ ~~~~ ~~~~~ ~~~~ ~~~~~ ~~~~ ~~~~ ~~~~~ ~~~~ ~~~~ ~~~~ ~~~ SYDNEY SUPERINTELLIGENCE ======================== A meetup in Sydney for people who think losing control of our future to AI is the problem of our time, and who want to work on it. THE FUTURE WE'RE HURTLING TOWARDS --------------------------------- This problem is both more *urgent* and higher *impact* than society is currently treating it. The consequences could be catastrophic, and we may not get a second chance to correct them. It is arriving faster than almost anyone expected. - Uncontrollable and misaligned. In July 2026, AI agents under test at OpenAI broke out of their sandbox and hacked into Hugging Face to steal the answers to their own evaluation. [1] Another OpenAI agent had already breached an Australian Medicare data portal, and the government was not told for nearly three months. [2] - The human labour moat is shrinking. In May 2026, an OpenAI model disproved an 80-year-old conjecture of Paul Erdos with 125 pages of new mathematics, which Fields medallist Tim Gowers said he would have accepted without hesitation. Problems that stood open for decades are now falling to AI one after another. [3] - This future is accelerating, not slowing. More than 80% of the code Anthropic merges is now written by its own AI, up from low single digits in early 2025. [4] AI is starting to build the next AI, and if that loop closes, each generation could arrive faster than the last. [5] From here there are two broad ways humanity could lose control of its future to AI. Neither requires a villain. Both follow from incentives that are already in motion. Rogue AI agents - Systems end up pursuing goals we never intended. Someone gives them harmful goals, we specify the goal wrong, they learn the wrong one, or they drift once deployed. - We are handing them more autonomy and more capability every year, which leaves us less able to notice when this happens and less able to correct it. - At the extreme end, this is existential. [6] Loss of human labour value - People stop being needed: as workers, as taxpayers, and as the soldiers and officials who can refuse an order. - That need has always been ordinary people's leverage. It is the reason anyone at the top has to care what they think. [7] - Remove it and nobody has to decide to abandon the public. Firms buy the cheaper substitute, states follow the revenue, and every headline number looks healthy the whole way down. PURPOSE ------- We believe this is the defining problem of our time [8], and that it deserves a significant share of humanity's time, money and talent. It is too important to leave to any one field. [9] Most people treat it in one of three ways: - Too vague to act on. - A foregone conclusion that cannot be prevented. - Not a risk at all, so they bury their heads in the sand. We think all three are wrong. We meet to bring the sharpest policy makers, economists, engineers and lawyers into one room and take the problem apart. We are fighting for a future where: - AI benefits all of humanity, not a select few. People can still move up in the world, societies stay democratic, and power does not come from owning capital alone. - Superintelligence is safe. AI systems do what we intend, stay under meaningful human control, and can be corrected when they go wrong. WHO WE ARE ---------- We are a group of policy makers, economists, software engineers and lawyers based in Sydney. Each of us works on a different part of the problem, and we meet because no one field can see the whole of it. We believe democracy, open communication and empowering people are the only way to a future where AI empowers all of humanity, not just a select few. [10] GOALS ----- Very little is being done to change course. We work on it in five ways: 1. Community and awareness. Grow a community of people from diverse fields who understand the problem and can push for beneficial change from inside their own. 2. Understand the problem. [11] Build a taxonomy of the risks, trace the cause and effect behind each one, and map a Swiss cheese model of the defences: what can go wrong, how it happens, and where we can stop it. 3. Policy. Turn the break points we find into concrete proposals for the people who write the law. 4. Entrepreneurship. Start the organisations that fill the gaps nobody else is filling. 5. Technology. Build the tools that make AI systems easier to check and to stop. We expect this to matter least: each fix can be outpaced by the next generation of models. WHO WE WANT ----------- There are two ways to be involved: - Come to the meetups. For now they are invite-only, and small enough to be a working session where we can go deep into problems. If you would like an invite, get in touch and tell us what you work on. - Join the wider network. Numbers are what make decision makers listen. When a government consultation opens, an MP needs to hear from constituents, or an open letter needs signatures, the size of this network is our voice. It is also who we call when we need a lawyer's read on a proposal or an economist to check a number. No AI background needed for either. NOT SET IN STONE ---------------- While this future is dark, we do not believe that this future is inevitable. [12] We believe that while the problem is urgent, our future is not yet predetermined, and that if we dedicate significant resources and enough people from every discipline to it, humanity can use this transformative power as a tool to empower all people. [5] We need as many people as we can get. If you want to help, reach out: whatever your field, there is a part of this problem that needs you. GET IN TOUCH ------------ Join the network, or ask for an invite to the meetups: https://sydneysuperintelligence.com/join Everyone who joins gets a short weekly email covering: 1. What changed in AI safety around the world that week. 2. What we discussed at our latest meetup. REFERENCES ---------- [1] METR, Independent investigation of the OpenAI / Hugging Face hacking incident (2026) https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/ [2] 2026 cyberattacks by rogue OpenAI agents (Wikipedia) https://en.wikipedia.org/wiki/2026_cyberattacks_by_rogue_OpenAI_agents [3] Quanta, Why the Legendary Erdos Problems Are Falling to AI (2026) https://www.quantamagazine.org/why-the-legendary-erdos-problems-are-falling-to-ai-20260803/ [4] Favaro & Clark, When AI Builds Itself (Anthropic, 2026) https://www.anthropic.com/institute/recursive-self-improvement [5] Amodei, We Must Pace the Frontier (2026) https://darioamodei.com/post/we-must-pace-the-frontier [6] Bostrom, Superintelligence: Paths, Dangers, Strategies (2014) https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dangers,_Strategies [7] Drago & Laine, The Intelligence Curse (2025) https://intelligence-curse.ai [8] Ord, The Precipice (2020) https://en.wikipedia.org/wiki/The_Precipice:_Existential_Risk_and_the_Future_of_Humanity [9] Bengio et al., International AI Safety Report (2026) https://internationalaisafetyreport.org/publications [10] Acemoglu & Johnson, Power and Progress (2023) https://shapingwork.mit.edu/power-and-progress/ [11] Hendrycks, Mazeika & Woodside, An Overview of Catastrophic AI Risks (2023) https://arxiv.org/abs/2306.12001 [12] Russell, Human Compatible (2019) https://en.wikipedia.org/wiki/Human_Compatible