Noetix HERON predicts how a robot action may change the scene before execution

A robot planner can compare predicted outcomes before sending a motion to hardware. That is useful when an early mistake would affect every later step.

Prediction quality has to be checked against the real scene after execution. A convincing video prediction can still contain a wrong trajectory or contact time.

Noetix says HERON is jointly pretrained on real-robot data, reconstructed simulation and human egocentric video. A unified Action Encoder converts heterogeneous controls into action tokens so full-action, partial-action and action-free clips can contribute to training.

The published architecture combines a frozen language branch through Mixture-of-Transformers with Mixture-of-Experts capacity across different diffusion noise stages. Those design choices are model evidence, not proof that every predicted future is physically correct.

HERON reports an overall WorldArena 1.0 Track 1 score of 75.80. Noetix also reports stronger interaction quality than trajectory accuracy, which matters because a future video can look plausible while placing an end effector on the wrong path.

For a robot, trajectory error can decide whether a gripper reaches an object, misses contact or strikes a fixture. The next useful benchmark is whether selecting actions with HERON reduces real physical failures under a fixed closed-loop protocol.

An offline benchmark is useful for model iteration. A hardware test should measure whether HERON reduces physical failures on a fixed task set.

Planning latency should also be reported because slow prediction can limit use on fast production cycles.