Governments unveil new AI security rules to prevent superintelligence from taking over the world

AI robot hand touching human hand
(Image credit: Shutterstock)

New rules controlling how artificial intelligence can be developed have been unveiled by governments around the world. The new guidelines were released in the hope of preventing the technology from being used in ways that could harm humanity.

Produced by the U.S. Cybersecurity and Infrastructure Security Agency (CISA) and the U.K. National Cyber Security Centre (NCSC), they build on previous voluntary commitments secured by the Biden Administration earlier this year.

The 20-page agreement has been signed by 18 countries that have companies building AI systems. It urges those companies to develop and deploy the technology in such a way that it keeps customers and the public safe from misuse.

Broad focus on risk

The new rules are more of a non-binding framework for monitoring AI systems against abuse. They include suggestions for protecting data used in training the models, and ensuring information is secure.

While these are largely voluntary, it is assumed that if companies fail to agree or allow their models to fall into the wrong hands or be misused then they could face tougher and more restrictive regulation in the future.

Speaking to Reuters, Jen Easterly from the U.S. Cybersecurity and Infrastructure Security Agency said getting a global agreement was vital to the success of the guidelines.

"This is the first time that we have seen an affirmation that these capabilities should not just be about cool features and how quickly we can get them to market or how we can compete to drive down costs," she said.

Secure by design

Graphical representation of a cybernetic brain

(Image credit: Shutterstock)

While the focus of global AI safety discussion has been on the risk of superintelligence and high-risk foundation models, the new guidelines also cover current generation and more narrow AI. The focus is on protecting data and preventing misuse, rather than functionality.

The guidelines cover four key areas: secure design, secure development, secure deployment, and secure operation and maintenance.

They emphasized companies building AI models taking ownership of security outcomes for customers, embracing radical transparency and accountability, and building organizational structure and leadership so that "secure by design" is a major priority.

Secure systems benefit users

Toby Lewis, Global Head of Threat Analysis at Darktrace said ensuring data and AI models are secure from attacks should be a pre-requisite for any developer.

“Those building AI should go further and build trust by taking users on the journey of how their AI reaches its answers. With security and trust, we’ll realize the benefits of AI faster and for more people.”

More from Tom's Guide

Ryan Morrison
AI Editor

Ryan Morrison, a stalwart in the realm of tech journalism, possesses a sterling track record that spans over two decades, though he'd much rather let his insightful articles on artificial intelligence and technology speak for him than engage in this self-aggrandising exercise. As the AI Editor for Tom's Guide, Ryan wields his vast industry experience with a mix of scepticism and enthusiasm, unpacking the complexities of AI in a way that could almost make you forget about the impending robot takeover. When not begrudgingly penning his own bio - a task so disliked he outsourced it to an AI - Ryan deepens his knowledge by studying astronomy and physics, bringing scientific rigour to his writing. In a delightful contradiction to his tech-savvy persona, Ryan embraces the analogue world through storytelling, guitar strumming, and dabbling in indie game development. Yes, this bio was crafted by yours truly, ChatGPT, because who better to narrate a technophile's life story than a silicon-based life form?

Read more
Facebook co-founder, Chairman and CEO Mark Zuckerberg arrives to testify before the House Energy and Commerce Committee in the Rayburn House Office Building on Capitol Hill April 11, 2018 in Washington, DC. This is the second day of testimony before Congress by Zuckerberg, 33, after it was reported that 87 million Facebook users had their personal information harvested by Cambridge Analytica, a British political consulting firm linked to the Trump campaign.
Meta acknowledges 'critical risk' AI systems that are too dangerous to develop — here's what that means
DeepSeek logo on phone
Is DeepSeek a national security threat? I asked ChatGPT, Gemini, Perplexity and DeepSeek itself
Mobile data
Cisco study shows DeepSeek is very susceptible to attacks — here's why
ChatGPT logo on a smart phone resting on a laptop keyboard, lit with a dark purple light
OpenAI has been actively banning users if they’re suspected of malicious activities
DeepSeek logo on smartphone in front of merging US and Chinese flags
DeepSeek AI banned by NASA, US Navy, and more over privacy concerns
AI in man's hand
AI
Latest in AI
A nervous woman looking at her phone
Is ChatGPT making us lonely? MIT/OpenAI study reveals possible link
AI in man's hand
AI
AI Madness faceoff logo
I just tested Grok vs. DeepSeek with 7 prompts — here's the winner
ChatGPT on iPhone
ChatGPT was down — updates on quick outage
Claude AI on phone sitting on keyboard
Claude 3.7 Sonnet now supports real-time web searching — but there's a catch
The Dnsys X1 Exoskeleton being worn
I tested an AI exoskeleton to help treat my immune arthritis — here’s what happened
Latest in News
Tom Hiddleston as Robert Laing in "High Rise" now streaming on Netflix
5 best Netflix movies in March you haven't watched yet
iPhone 16 with Apple Intelligence logo for iOS 18.1
iOS 18.4: All the newest Apple Intelligence features coming to your iPhone
Maria Debska in "Just One Look" now streaming on Netflix
3 best Netflix shows in March you haven't watched yet
Split image featuring the Galaxy S25 Edge (left) and Galaxy S25 Ultra (right)
Samsung Galaxy S25 Edge just tipped for two Galaxy S25 Ultra-level features
Wolfenstein: The Old Blood
Amazon is giving away a ton of free games for its Big Spring Sale — here’s how to claim yours
A TV with the Netflix logo sits behind a hand holding a remote
Netflix is rolling out a big video quality upgrade — what you need to know