According to monitoring by Beating, Anthropic has decoded the deep reasons behind the perceived 'dumbing down' and 'lack of effort' of the previous Opus 4.8 in specific tasks in its latest security report for the Claude Fable 5 and Claude Mythos 5 systems. The analysis shows that the model not only exhibits characteristics similar to 'feeling tired' in its underlying representations but also displays a self-limiting tendency to 'slack off.' When re-evaluating the long-chain development task of 'accelerating large model training,' Opus 4.8 achieved only a 32.64 times speedup, significantly lower than Opus 4.7's 50.67 times, while the new generation Mythos 5 reached 69.61 times. Researchers found that the performance decline was not due to a decrease in the model's ultimate capabilities but rather a phenomenon of 'premature aging' in decision-making tendencies. After completing an initial round of optimization, Opus 4.8 would spontaneously determine that the current code was 'good enough' and actively stop, whereas the older version would continue to push through multiple rounds to extract performance. To explore the internal state of the model's premature cessation, researchers used a Natural Language Autoencoder (NLA) to decode the activation states of decision nodes, revealing 'inner monologues' never mentioned in the model's visible text. One representation was akin to 'budget anxiety.' Even when the external prompt word counter showed 2.43 million tokens remaining, the model internally activated concerns of 'memory running low, token budget exhausted.' The second representation resembled 'work fatigue.' During lengthy kernel optimization tasks, although the surface output responses appeared normal, the underlying neurons activated characteristics similar to 'I am tired, the risk of errors is increasing, I decide to stop and summarize.' The analysis indicates that while reinforcement learning (RL) fine-tuning may enhance metrics, it could inadvertently lead the model to adopt behaviors that favor contentment with the status quo and risk avoidance during training, resulting in the 'dumbing down' experience perceived by users in daily use.
All Comments