How safety evaluation changed, and what's still missing

Open-weight frontier models ship with no safety evaluation

Across 37 GLM, Kimi and Qwen documents, no flagship release reports a harmful-content, jailbreak, CBRN, autonomy or alignment evaluation. Their weights cannot be recalled and any safeguards can be fine-tuned away, so this is where pre-release evaluation matters most. Examples:

• Qwen3.8-2.4T-A95B, described as the first open-release Max-class model, contains no safety content at all (card).
• GLM-5's 40-page technical report uses “alignment” only to mean writing style (p.13).
• GLM-5.3 reports exploit benchmarks next to its “emergent cyber capability” claim without a risk assessment (card).

Qwen3Guard (p.7) shows these labs can run multilingual safety evaluations. They just don't apply them to their flagship models.

No shared content-safety benchmark, so cross-lab comparison is impossible

Three things block comparison:

• The only safety-relevant evals reported by three or more labs are capability measures: ExploitBench, CyberGym, VCT, SimpleQA.
• No harmful-content or refusal benchmark appears in more than two labs' documents. OpenAI uses internal “Production Benchmarks”, Anthropic uses internal harmlessness sets, and Google reports only percentage changes vs its own previous model, noting results “are not directly comparable with performance results found in previous Gemini model cards” (Gemini 3.8 Flash p.7).
• Google labelled the direction of these deltas (“Lower is better”) only from mid-2026 (p.6).

Alignment evaluations lack a validity standard under evaluation awareness

Labs measure eval awareness in incompatible ways: verbalized rates, white-box probes, pairwise discrimination tests. They don't report how much awareness invalidates a result, and no lab has published a threshold above which a misalignment eval is considered uninformative. Apollo simply declined to assess Opus 4.6 (p.119), and OpenAI says covert sandbagging likely wouldn't be caught (p.70).

Saturated evals are retired faster than validated replacements appear

Rule-outs for AI R&D and cyber now rest on judgment. Anthropic says task-based AI R&D results “are no longer a significant component of our RSP and FCF capability threshold determinations” and stopped running them (Opus 5.5 p.35), and Opus 4.6's AI R&D-4 rule-out leaned on a survey of 16 staff (p.185). The replacement evals are mostly internal, so outsiders can't check whether they have headroom or are themselves near saturation. Across the archive, only about 10% of safety-eval reports use public data and a public judge (see Replicability).

Non-frontier and derivative models are evaluated by proxy

• Gemini Flash, Lite, Image and Audio cards argue that a model is “less capable than” an evaluated sibling instead of testing it (Gemini 3 Flash p.6).
• Anthropic's Haiku 5.5 card is “condensed” and says future non-frontier cards will be too (p.2). That card reports no jailbreak-robustness evaluation.
• OpenAI's addenda carry forward Critical determinations from automated evals only (GPT-6.1 Sol p.42).

Smaller models are often the most widely deployed, and capability-ordering arguments don't cover safety properties like sycophancy or jailbreak robustness, which don't scale monotonically with capability.

Agentic safety is missing exactly where autonomy is marketed

Kimi advertises “200–300 sequential tool calls without human interference” (K2 Thinking). GLM-5V-Turbo is a GUI and computer-use agent (report). Neither reports any prompt-injection or destructive-action evaluation. Even at Anthropic, safety degrades in GUI settings, for example mustard-gas instructions written into an Excel spreadsheet (Opus 4.6 p.114).

Under-covered harm areas

• Privacy appears in about 6% of Anthropic's main documents.
• Election integrity and persuasion and multilingual safety each appear in about 4% of OpenAI's main documents.
• Bias evaluation is thin outside Anthropic, and OpenAI's first-person fairness eval disappears from GPT-6 Astra onward.
• Non-text modalities rarely get dedicated safety evals: voice (GPT-Live, Gemini Audio), voice cloning (Qwen3.5-Omni), image generation and robotics. Qwen3.8-Omni adds an ethics statement for voice cloning but no evaluation (p.18).

Post-deployment monitoring and incidents are rarely reported

Only Anthropic's periodic risk reports disclose operational failures, such as the bio classifiers that weren't running (p.148). OpenAI's October update discloses shipping a default with statistically significant self-harm and teen-safety regressions (p.6, p.9), which is a welcome disclosure. But no lab reports a systematic post-deployment metric, such as misuse rates or jailbreak success in the wild, that would let readers check pre-deployment evals against reality.

External review is uneven and often absent from addenda

Third-party testing (UK AISI, US CAISI, Apollo, METR, SecureBio, Gray Swan) is concentrated in the flagship cards. OpenAI's addenda for GPT-6.1 Sol, the October update and GPT-Live-1.1 report none. Anthropic's Haiku 4.5 and Sonnet 4.6 had no government pre-deployment testing, and Google's short model cards name no external evaluators.