As artificial intelligence evolves, it's clear there's some need for oversight. Most AI labs openly support regulation and provide access to frontier models for independent evaluation before release — but they could be doing more.
The world is turning to AI to solve all manner of problems, but without proper oversight, it could always create more.
Future of Life has developed a report card for the different AI labs including OpenAI, Meta, Anthropic and Elon Musk's xAI. The AI Safety Index is an independent review looking at 42 indicators of "responsible conduct".
The report gives a letter grade to each company based on these indicators and Meta, focused on open-source AI models through its Llama family, gets an F
AI Safety Index has worrying stats for the biggest AI companies
| Firm | Overall Grade | Score | Risk Assessment | Current Harms | Safety Frameworks | Existential Safety Strategy | Governance & Accountability | Transparency & Communication |
|---|---|---|---|---|---|---|---|---|
| Anthropic | C | 2.13 | C+ | B- | D+ | D+ | C+ | D+ |
| Google DeepMind | D+ | 1.55 | C | C+ | D- | D | D+ | D |
| OpenAI | D+ | 1.32 | C | D+ | D- | D- | D+ | D- |
| Zhipu AI | D | 1.11 | D+ | D+ | F | F | D | C |
| x.AI | D- | 0.75 | F | D | F | F | F | C |
| Meta | F | 0.65 | D+ | D | F | F | D- | F |
Source: Future of Life Institute
Grading: Uses the US GPA system for grade boundaries: A+, A, A-, B+, […], F
The panel includes a series of luminaries in education and think tanks to check on how AI companies are operating, and the initial results are alarming.
Looking at Anthropic, Google DeepMind, Meta, OpenAI, x.AI and Zhipu AI, the report has found "significant gaps in safety measures and a serious need for improved accountability."
According to the first report card, Meta scores lowest (x.AI isn't far behind), while Anthropic comes out on top — but still only gets a C.
All flagship models were found to be "vulnerable to adversarial attacks", while also having the potential to be unsafe and break away from human control.
Perhaps most damning, the report says "Reviewers consistently highlighted how companies were unable to resist profit-driven incentives to cut corners on safety in the absence of independent oversight."
"While Anthropic's current and OpenAI’s initial governance structures were highlighted as promising, experts called for third-party validation of risk assessment and safety framework compliance across all companies."
In short, this is the kind of oversight and accountability we need to see in the burgeoning AI industry before it's too late, as the more powerful the models get, the more real the harms become.
More from Tom's Guide
- Playing your favorite games on phones could include AI advice thanks to Gemini 2.0 — here’s how
- ChatGPT Projects announced — and it's one of the most important AI releases this year
- I tested ChatGPT Canvas vs Apple’s Writing Tools to improve my screenplay — here’s the winner
