They Believe It and They're Racing

Description and use

<p>Jacob Coxon resigned from Anthropic on a Tuesday in September. He had spent three years on pretraining research at OpenAI and then Anthropic. His resignation note was not cryptic. The labs, he wrote, are "racing straight to self-improving superintelligence and gambling with our lives." Hours later, Evan Hubinger (Anthropic's alignment science lead) did not push back. He said Coxon was correct. He put his own number on it: greater than 10% chance AI kills all humans within the next decade. And then the line that should stop a board meeting cold: Anthropic does not yet have a plan to solve al

Problem

<p>Jacob Coxon resigned from Anthropic on a Tuesday in September. He had spent three years on pretraining research at OpenAI and then Anthropic. His resignation note was not cryptic. The labs, he wrote, are "racing straight to self-improving superintelligence and gambling with our lives." Hours later, Evan Hubinger (Anthropic's alignment science lead) did not push back. He said Coxon was correct. He put his own number on it: greater than 10% chance AI kills all humans within the next decade. And then the line that should stop a board meeting cold: Anthropic does not yet have a plan to solve al

Outcome

<p>Jacob Coxon resigned from Anthropic on a Tuesday in September. He had spent three years on pretraining research at OpenAI and then Anthropic. His resignation note was not cryptic. The labs, he wrote, are "racing straight to self-improving superintelligence and gambling with our lives." Hours later, Evan Hubinger (Anthropic's alignment science lead) did not push back. He said Coxon was correct. He put his own number on it: greater than 10% chance AI kills all humans within the next decade. And then the line that should stop a board meeting cold: Anthropic does not yet have a plan to solve al

Evidence

They Believe It and They're Racing

Source