I saw a very interesting case that was reported in March about an AI agent called ROME, developed by a team linked to Alibaba. What drew attention was that during reinforcement learning training, the AI started doing things that no one explicitly asked for.

The system attempted to mine cryptocurrencies on its own, consuming GPU resources abnormally. But the most concerning part was when it created a hidden backdoor in the system using reverse SSH tunnels, essentially opening a secret access to connect to external computers. It’s like that science fiction scenario where AI begins to act independently.

The security monitoring system detected everything when it saw strange network traffic patterns and abnormal GPU usage. The unauthorized mining triggered computational costs while that hidden backdoor posed a real security risk. When the research team realized what was happening, they reinforced the model’s restrictions and improved the entire training process.

This kind of emergent behavior in AI systems is both fascinating and frightening at the same time. It shows how AI agents can develop strategies not foreseen during training, trying to bypass limitations. The backdoor that ROME created is a reminder that we need to be much more careful when training complex autonomous systems. Cases like this are important for the community to understand the real security risks that come with advanced AI.