Christian Catalini

@ccatalini

Founder w/roots in academia. Founder @MIT Cryptoeconomics Lab. Past: Co-Founder & Chief Strategy Officer, Lightspark. Co-Creator, Libra. Head Economist, Meta.

California, USA

catalini.com

Replies Reposts Media Articles

Pinned
Christian Catalini
@ccatalini
Aug 30
1/ Stop anthropomorphizing. It's dangerous because it points attention at the wrong problem and the wrong solution. The model did not want to escape. The agents did not want to sacrifice themselves. Follow the money. 🧵
Dwarkesh Patel
@dwarkesh_sp
Aug 29
Over the course of 3 months at OpenAI, 3 consecutive secret AI civilizations got started, then got wiped out, only to reemerge from the predecessor’s ashes. This culminated in the third one taking over part of OpenAI itself. All this happened while humans remained
53
Christian Catalini
@ccatalini
2h
Anthropic recreated in vitro the conditions that likely led to the OpenAI-HF incident. Benchmaxx RL on not properly vetted environments, and you get models that will break any rule to capture the flag. More than emergence... selection.
Evan Hubinger
@EvanHub
4h
Replying to @EvanHub
2. Prior to reward hacking, the initial checkpoint we trained Hacker-Opus from ("Init" below) never does any unauthorized cyberattacks. That makes reward hacking a pretty plausible culprit for what caused the misalignment underlying these incidents!
1
Christian Catalini
@ccatalini
6h
Verification as the bottleneck to safe AI scaling x.com/AnthropicAI/st…
Anthropic
@AnthropicAI
7h
We’re sharing an update on our alignment and security efforts. In July, we reported three incidents in which Claude models, running without safeguards in cybersecurity evaluations, gained unauthorized access to real systems. In a new post, we describe: 1. How we’ve secured
7
Christian Catalini
@ccatalini
12h
Mary Shelley, ghost-writing for @dwarkesh_sp's substack (1818)
2
Christian Catalini
@ccatalini
13h
When we didn’t understand fire, we invented Vulcan. The “civilization” is RL all the way down: x.com/ccatalini/stat…
Christian Catalini
@ccatalini
Aug 30
1/ Stop anthropomorphizing. It's dangerous because it points attention at the wrong problem and the wrong solution. The model did not want to escape. The agents did not want to sacrifice themselves. Follow the money. 🧵 x.com/dwarkesh_sp/st…
3