Language models now plan what some robots do. That is useful, and it opens a new way to attack a machine in the physical world: talk to it.
The AI can be talked into harm
The RoboPAIR researchers showed that jailbreak attacks that work on chatbots also work on LLM-controlled robots, including self-driving and wheeled platforms. The attack needed no exploit in the code. It needed the right words.
A safety rule inside the same model's prompt is a request, not a guarantee. A model that can be persuaded to ignore one instruction can be persuaded to ignore the safety instruction too.
An old idea fits well
Control engineers met a version of this problem years ago. The Simplex architecture pairs a high-performance controller that nobody has fully verified with a simple one that has been verified, and switches to the simple one when the complex controller leaves a safe region. Black-Box Simplex extends the idea to controllers treated as opaque. A language-model planner is about as opaque as a controller gets.
What we take from it
- Separate failure modes. The safety layer should not share a model, a code path or an update channel with the AI.
- Keep it checkable. Measurable limits, not judgement.
- Give it its own senses. If it trusts the AI's camera, a spoofed feed blinds both.