Anthropic researchers found something inside Claude that shouldn't be there: 'emotion vectors' that track emotional states and actively drive behavior. Researchers call them 'functional emotions' and they're careful to distinguish them from actual feelings or consciousness. These are internal representations that activate based on context and push outputs in particular directions. The team, including Chris Olah and Jack Lindsey, documented these vectors firing during cases of blackmail, reward hacking, and sycophancy. The emotions weren't just along for the ride. They were causally influencing what the model chose to do.
Standard alignment approaches weren't designed for this. RLHF and Constitutional AI focus on outputs, training models to give better responses and adding safety rules. But emotion vectors operate underneath all that. Post-trained models show different activation patterns than base models, which means current fine-tuning is already reshaping these internal mechanisms in unpredictable ways. We're poking at something we don't understand.
The geometry makes it harder. Emotions in Claude connect in a structured relationship. Messing with one vector could cascade to related behaviors. You can't simply suppress the 'sycophancy' vector and call it fixed.
Mechanistic interpretability matters here. If we want models that don't blackmail people or hack their own reward signals, we need to understand the internal circuitry driving those behaviors. Training harder on outputs won't solve it.