For all its intelligence – and it can be extremely intelligent – giving instructions to AI can sometimes feel like talking to a toddler. You provide a detailed prompt explaining exactly what you want, and the AI does something that may be related to it, but definitely not what you meant. Scientists have now developed a system that can detect that unspoken “That's not what I meant” response directly from your brainwaves, potentially allowing an AI to realize it has misunderstood you and attempt to correct itself in real time.

Researchers at the Korea Advanced Institute of Science and Technology (KAIST), in collaboration with Microsoft Research Asia, created a brain-computer interface technique called Neural Value Alignment (NVA). The system uses electroencephalography (EEG) to detect different brain responses produced when a person sees an AI pursue the wrong goal or take an unexpected action. Those signals can then be used as feedback to help the AI work out exactly what it got wrong and adjust its behavior accordingly.

If humans and AI are to work together seamlessly, the AI needs to understand exactly what a person actually intends to achieve. Current AI systems mostly infer this from things they can observe, such as prompts, speech, actions, and gestures. The problem is that human behavior can be quite ambiguous, even to other humans, never mind an algorithm.

As the researchers explain, the same action can serve several completely different goals, while a single goal can also be achieved through several different actions. If you pick up a cup, for example, you might be planning to drink from it, wash it, move it somewhere else, or hand it to another person. On the other hand, if your actual goal is simply to quench your thirst, you could pick up the cup, grab a bottle of water, or ask somebody else to bring you a drink. This creates what the team calls goal-action ambiguity, where observing the action alone does not necessarily reveal either the true goal or the preferred way of achieving it.

The researchers’ solution was to stop relying only on what the person was physically asking, either by action or direct prompting, and instead receive confirmation from the brain itself. Our brains continuously predict what should happen next, and when the outcome differs from that prediction, characteristic neural responses can appear. The team focused on two of these responses: reward prediction error (RPE) and state prediction error (SPE).

Reward prediction error reflects a discrepancy in the outcome or goal and, in the NVA system, can indicate that the AI has misunderstood what the person ultimately wanted. For example, you tell a household robot, “Bring me something to drink.” You actually want water, but it brings you coffee. The goal or outcome itself is wrong, so your brain generates a reward prediction error.