ninazino

Mossing and others, both at OpenAI and at rival firms including Anthropic and Google DeepMind, are ... studying them [LLMs] as if they were doing biology or neuroscience on vast living creatures—city-size xenomorphs that have appeared in our midst.

Anthropic and others have developed tools to let them trace certain paths that activations follow, revealing mechanisms and pathways inside a model much as a brain scan can reveal patterns of activity inside a brain. Such an approach to studying the internal workings of a model is known as mechanistic interpretability. “This is very much a biological type of analysis,” says Batson. “It’s not like math or physics.”

Anthropic invented a way to make large language models easier to understand by building a special second model (using a type of neural network called a sparse autoencoder) that works in a more transparent way than normal LLMs. This second model is then trained to mimic the behavior of the model the researchers want to study.

Creating a model that behaves in predictable ways in specific scenarios requires making assumptions about what the inner state of that model might be in those scenarios. But that only works if large language models have something analogous to the mental coherence that most people do.

And that might not be the case.

...
Another possible solution ... Instead of relying on imperfect techniques for insight into what they’re doing, why not build an LLM that’s easier to understand in the first place?

https://www.technologyreview.com/2026/01/12/1129782/ai-large-language-models-biology-alien-autopsy/