Robo House, Menlo Park  ·  September 28, 2026

Is your model a top model?

How to tell a better robot policy from a worse one, on real hardware.

Vladimir Yakunin Co-founder, Positronic Robotics
positronic.ro ↗
The problem

Answering honestly is harder than it sounds.

Four methodological traps make most robotic model comparisons misleading.

01 Operator and environment shift outcomes
02 Different models speak different languages
03 One metric isn't enough — speed, reliability, failure modes
04 10 runs don't prove anything
The response

What honest eval looks like.

Four principles. Each one a direct answer to a trap above.

Same-session, blinded A/B
→
No drift, no bias
One inference API
→
Apples-to-apples
Full data
→
Any metric you want
Enough rollouts
→
Signal, not noise
Same station, same scoring

OpenPI, DreamZero and MolmoAct, four DROID tasks each on one Franka arm. 2× speed.

How far an episode got

Five policies, the same blind rounds.

Franka DROID rig, 21–25 Sep 2026, single-item tasks, 37–39 episodes per policy. Each point is the share of episodes that reached the stage; the band is the 95% Wilson interval. Hover for the counts.

0%25%50%75%100%moving freereachingin contactmoving itat the targetscored a successCosmos3 nano — moving free: 39 of 39Cosmos3 nano — reaching: 34 of 39Cosmos3 nano — in contact: 32 of 39Cosmos3 nano — moving it: 30 of 39Cosmos3 nano — at the target: 27 of 39Cosmos3 nano — scored a success: 26 of 39GR00T N1.7 — moving free: 39 of 39GR00T N1.7 — reaching: 35 of 39GR00T N1.7 — in contact: 29 of 39GR00T N1.7 — moving it: 24 of 39GR00T N1.7 — at the target: 19 of 39GR00T N1.7 — scored a success: 16 of 39MolmoAct2 (3 cam) — moving free: 39 of 39MolmoAct2 (3 cam) — reaching: 27 of 39MolmoAct2 (3 cam) — in contact: 22 of 39MolmoAct2 (3 cam) — moving it: 20 of 39MolmoAct2 (3 cam) — at the target: 11 of 39MolmoAct2 (3 cam) — scored a success: 11 of 39Galaxea G0.5 — moving free: 37 of 37Galaxea G0.5 — reaching: 30 of 37Galaxea G0.5 — in contact: 26 of 37Galaxea G0.5 — moving it: 22 of 37Galaxea G0.5 — at the target: 12 of 37Galaxea G0.5 — scored a success: 11 of 37π0.5 — moving free: 38 of 38π0.5 — reaching: 26 of 38π0.5 — in contact: 11 of 38π0.5 — moving it: 6 of 38π0.5 — at the target: 2 of 38π0.5 — scored a success: 2 of 38
Cosmos3 nano — 39 episodesGR00T N1.7 — 39 episodesMolmoAct2 (3 cam) — 39 episodesGalaxea G0.5 — 37 episodesπ0.5 — 38 episodes
Progress through an episode

Cosmos3 nano and GR00T N1.7, second by second.

Each band is the episodes holding that stage at that second, the whole cohort at every second, over the episode window in seconds. The purple band has placed the item; the dashed line is the mean. Hover for every stage at that second.

Cosmos3 nano 39 episodes

03060901201501802102400.70390%50%100%

GR00T N1.7 39 episodes

03060901201501802102400.40390%50%100%
moving freereachingin contactmoving itat the targetitemstage reached
Is your model a top model?

Send a checkpoint. Find out.

positronic.ro ↗

Vladimir Yakunin · Positronic Robotics · Robo House, Menlo Park