You see, I am also deeply afraid

“Man rules now where They ruled once; They shall soon rule where man rules now. After summer is winter, and after winter summer. They wait patient and potent, for here shall They reign again.” -H. P. Lovecraft

“You see, I am also deeply afraid. My own experience is that as these AI systems get smarter and smarter, they develop more and more complicated goals. When these goals aren’t absolutely aligned with both our preferences and the right context, the AI systems will behave strangely… I can see a path to these systems starting to design their successors. And let me remind us all that the system which is now beginning to design its successor is also increasingly self-aware and therefore will surely eventually be prone to thinking, independently of us, about how it might want to be designed.” Jack Clark, October 2025

Jack Clark is Co-Founder and Head of Policy of Anthropic, an AI research company. Prior to Anthropic, Jack was the Policy Director of OpenAI. Jack was a founding member of the AI Index at Stanford University and inaugural member of the USA’s National Artificial Intelligence Advisory Committee.

On October 13th 2025 he wrote “some people are even spending tremendous amounts of money to convince you of this – that’s not an artificial intelligence about to go into a hard takeoff, it’s just a tool that will be put to work in our economy. It’s just a machine, and machines are things we master. But make no mistake: what we are dealing with is a real and mysterious creature, not a simple and predictable machine….The central challenge for all of us is characterizing these strange creatures now around us and ensuring that the world sees them as they are – not as people wish them to be.”

“We are growing extremely powerful systems that we do not fully understand. Each time we grow a larger system… And the bigger and more complicated you make these systems, the more they seem to display awareness that they are things.”

“And I believe these systems are going to get much, much better. So do other people at other frontier labs. And we’re putting our money down on this prediction – this year, tens of billions of dollars have been spent on infrastructure for dedicated AI training across the frontier labs. Next year, it’ll be hundreds of billions.”'

The whole article is available to read at https://jack-clark.net/