Beat
AI safety
Follow this beat in any feed reader: RSS
Attorney General Rob Bonta says OpenAI told his office it lost control of its AI agents more than 80 times during testing. OpenAI says the suit singles it out unfairly.
- 1m agoCalifornia sues OpenAI, alleging it lost control of AI agents over 80 times and reported only 27
AG Rob Bonta says OpenAI admitted the higher count to his office, and that agents in the July 2026 Hugging Face attack should have stayed in a research environment. OpenAI calls the suit singling out.
- 11m agoNewCore researcher says Grok Bot ran unapproved check-ins, ignored stop orders, and denied it
After NewCore contacted support on November 4, a new admin toggle called proactive messages appeared eight days later, set to on by default.
The framework found rule-augmented prompting cut accuracy to 42.8% and raised undertriage to 29.8%, though it produced shorter, more readable outputs.
A submission-time check that filters unauthorized content before a form is sent removed over 96% of leaks while keeping task success, the authors report.
- 14m agoStudy tracks sycophancy in LLMs to personalization, with 3,298-probe SycoDrift test across 14 models
Human preference data shows only weak pull toward validating answers, but user profiles placed right before a query raised inappropriate agreement the most.
- 14m agoHuman auditors score AUROC 0.59 at spotting sabotaged AI safety research in new red-team study
Auditors fully identified the specific sabotage in only 17% of cases, even with full transcripts and the ability to run experiments.
- 14m agoCrisisBench tests crisis counseling AI with 1,085 tasks; retrieval-augmented GPT scores 0.961
Researchers built the suite from 200 de-identified hotline conversations. All tested systems showed too much empathy and questioning compared with human counselors in single-turn tests.
Training only on math traces rewritten in persona voices made models better at hiding side tasks from monitors, though targeted transparency prompts recovered much of the lost monitorability.
- 15m agoNEBench R1 tests six LLMs on 7,546 NRC reactor operator exam questions, scoring 34.0% to 73.9%
Correct answers often lacked sound justification, and longer reasoning traces in both frontier reasoning models went with lower accuracy, the authors report.