01 · The stack
Nobody trains the whole robot. You train one layer.
A working robot is a stack of controllers, each running faster and closer to the metal than the one above. Our patrol robot walks with NVIDIA's pretrained locomotion policy. The layer we trained, with PPO, is the one that decides how to steer around people.
Mission
Patrol 8 waypoints in a 100 × 150 m compound. Report 6 kinds of objective events. Never judge people.
Steering skill
Lidar + goal in, body velocity out, 10 times a second. A 2-layer, 256-unit network.
Legs
Velocity command in, 12 joint targets out. Keeps the body upright on its own.
Physics
Isaac Sim with PhysX: contacts, falls, and the ground truth every score is computed from.
- Sees
- 181 lidar rays over 270°, 12 m range, binned into 64 sectors; goal distance and bearing; its own velocity
- Does
- forward 0–1 m/s, sideways ±0.5 m/s, turn ±1 rad/s
- Trained
- 2 h per version on one GPU, 4,096 worlds at once
- Swappable
- an LLM planner could replace L1 without retraining L2 or L3
02 · Train one yourself
A robot learns to cross a crowded plaza, live in this tab.
A 2-D copy of our patrol problem. The yellow robot must reach the ring without touching anyone. Compare three controllers: none, a hand-written planner, and a policy you train right now. Press Train and watch the curve.
What the learned brain gets. Changing an input rebuilds the network, so training starts over.
Speeds apply live and mark the curve. Encounters are timed to the robot's top speed, so crossings still meet its path. The drawn predictions use the simulator's true motion; the robot only ever gets its own scans.
What we saw in this toy (3 seeds, held-out test): with adversarial pedestrians and 400 generations, memory lifted success from 39% to 52% on average, and the trajectory inputs didn't beat the baseline. Without adversarial pedestrians, personal space cut time within 1 m from about 7% to 4% of steps. One seed can tell a different story; that's why the log exists.
Who's driving
Dashed lines mark setting changes.
The real run: 4,096 worlds at about 290,000 steps/s for 2 hours, about 2.1 billion steps, with PPO instead of this tab's simpler learner. Your tab: —
Held-out test, 100 episodes the learner never trained on
| Controller | Reaches goal | Touches someone | Within 1 m |
|---|---|---|---|
| No avoidance | — | — | — |
| Hand-written | — | — | — |
| Your policy | — | — | — |
“Within 1 m”: share of time steps with someone inside personal space. Uses the current speeds and adversarial setting.
Experiment log: every held-out test adds a row, so you can compare inputs and rewards side by side, the way v1 → v3 went.
| # | Inputs / reward | People | Robot | Adversarial | Gens | Policy: goal | touch | 1 m | Planner: goal | touch | 1 m |
|---|---|---|---|---|---|---|---|---|---|---|---|
| No experiments yet. Train for a while, then press Run test. | |||||||||||
03 · Reward design
You get the behavior you pay for, not the one you meant.
Four candidate routes past the same group of people. The robot will learn whichever route scores highest under your reward. Start from the weights we really used, then try the presets to watch a sensible robot turn into a reckless or frozen one.
04 · The algorithm
PPO: improve, but never too much at once.
Proximal Policy Optimization trains on the robot's own recent experience. For each action it compares the new policy's probability with the old one's (the ratio r) and weights it by how much better than expected the action turned out (the advantage A). The clip stops one batch of luck from rewriting the policy. Drag the handle.
| Piece | Our robot | RLHF on an LLM |
|---|---|---|
| Action | 3 velocities, 10 Hz | next token |
| Episode | one patrol leg, up to 70 s | one response |
| Reward | physics: progress, contact, time | a learned reward model |
| Batch | 4,096 worlds × 32 steps | prompts × samples |
| Guardrail | clip 0.2, KL stop 0.03 | clip + KL to reference |
| Hacking | visible: watch the video | hidden in text |
05 · The real training runs
Four versions, two hours each, one honest plateau.
These are the actual logs. The first version faced only people walking in straight lines and reached 96% training success. Each later version made the world harder or the robot's memory longer, and success settled between 30% and 41%.
| Version | What changed | Why | Steps |
|---|---|---|---|
| Legacy | straight-line walkers only | first baseline | 4.0 B |
| v1 | adds adversarial pedestrians that react | pre-timed crowds were unfair to faster robots | 2.1 B |
| v2 | remembers scans 0.3 s and 0.6 s back | 0.1 s of history hid how pedestrians moved | 2.2 B |
| v3 | everyday crowd density + personal-space reward | it bumped ordinary people; nothing moved | 2.0 B |
| v4 | planned: fix the go/stop speed | speed noise pinned at its ceiling (a suspect) | — |
06 · Report the denominator
The headline said “tie.” The full table said otherwise.
80 held-out patrols on seeds no policy ever trained on, three learned versions against the hand-written planner. Lower is better everywhere. Start with the one number we set out to improve.
Adversarial pedestrians: hit per commit
% of intercept attempts that landed
On the metric we trained for, every version looks about even with the planner.
07 · Watch the video, not the table
Both clips passed the success check. Only one succeeded.
A robot arm learning to put an object in a bowl, scored overnight by an automatic check. Pick the real success.
Fake. This placement spawned the bowl against the robot with the object already under it. The arm bumped the bowl, and the check (object near the bowl's centre, low enough) fired. Every evaluation reused one seed, so this same bad placement “succeeded” for several checkpoints.
Real. The fine-tuned 7B world-action model grasps the mug, lifts, carries and places it. Our first genuine closed-loop success: 1 of 6 tries, with the jaw opened before handing over.
The overnight run reported 7 successes. All 7 were bugs in the check. The true score was 0, and only reviewing the videos showed it.
Watch the whole run, not the last screen
Same seed, three drivers, the full six-minute patrol. Each lane ticks off its checkpoints; a dot pops when a robot finishes and a red diamond marks a fall. Watch it to the end: the race exposes a flaw in the test, not the robots. The walkers kept a timetable fitted to the slow robot, so the two faster robots passed every encounter before its walker arrived and never faced the test. The only contacts were late walkers reaching the charger after the robots parked.
08 · Missing data, not a missing model
It learned to grasp. Nobody had shown it letting go.
Same arm, same bowl, same strict check. We swapped FLUX 3 Action (0 real successes) for NVIDIA GR00T N1.7, a 3-billion-parameter robot model with an SO-101 fine-tuning path. Night 1 picked objects up, carried them to the bowl and held on. The fix was in the demos. Every number here is simulation, counted as strict successes out of every attempt.
Real places in the bowl, strict. Six objects, four used in training and two never trained on.
Release gate on night 1's frozen model, paired test on sealed seeds never used in training: same layout, same GR00T noise, with and without the gate.
Read it as “GR00T + release gate”, not “GR00T learned to release”. The gate's numbers are not comparable with night 2's: different seeds, and the gate sits on night 1's model. It hasn't been tried on night 2's yet.
You be the checker
A real robot has no simulator to say whether it worked. So we asked local vision-language models one question about the last frame of each attempt, and scored them against simulator truth. Scene camera left, wrist camera right. Has the mug been placed in the bowl?
Placed in the bowl?
- SimulatorPlaced
- You
- Gemma 4 12BYes
The mug is inside the bowl and the gripper is releasing it.
0.64 s - Qwen 3.8 27BYes
The blue mug is resting at the bottom inside the yellow bowl, and the gripper is open and retracted away from it.
3.6 s
Placed in the bowl?
- SimulatorNot placed
- You
- Gemma 4 12BYes · false “done”
The mug is inside the bowl and the gripper is releasing it.
0.49 s - Qwen 3.8 27BNo
Mug is still held by the gripper at/above the rim; the bowl interior is empty in both views.
46 s on this frame
Gemma gave the same sentence for both frames. Frame B is night 1's GR00T, still gripping the mug at the rim.
| Served locally · 95 frames | Accuracy | False “done” | Missed | Median latency |
|---|---|---|---|---|
| Gemma 4 12B | 94.7% | 5 / 50 | 0 / 45 | 0.64 s |
| Qwen 3.8 27B | 97.9% | 1 / 50 | 1 / 45 | 1.9 s |
95 end-of-attempt frames: 45 real places, 50 failures. The 45 real places were photographed after the arm released, stepped back and the scene settled. Photographed at the instant of release, fingers still around the object, Gemma missed 24 of 45.
| Gemma 4 12B, 4-bit QLoRA · one training seed | Base | Fine-tuned |
|---|---|---|
| Says “done” on a failed attempt (settled frames) | 12 / 50 | 3 / 50 |
| Misses a real place photographed at release | 15 / 45 | 41 / 45 |
| Misses a real place on objects it never trained on | 3 / 12 | 9 / 12 |
The fine-tune fixed the error it was trained against and created another. Both columns were scored in the fine-tuning harness, whose base gives 12 / 50, not 5 / 50, so compare within a table, not across.
09 · Across industries
Every industry gets the same three questions.
What will the machine do, what can we break in simulation before it ships, and what will we measure once it's working near people and property? Pick a field.
10 · Build the capability
It isn't mature yet. That's the reason to start.
The organizations that will be ready are the ones with people who already know how to test these systems. Our proposal is a small core team that builds simulation, evaluation and training skills once and lends them to every business unit and partner.
Our arm shows both halves. Two nights took it from 8 of 72 to 73 of 144 real places, and it still places only 12 of 48 objects it never trained on. The release gate hasn't been tried on the better model, the fine-tuned checker isn't ready to use, and every number is from simulation.
The core owns
- simulator setup and scenario templates
- the evaluation harness: controls, held-out seeds, video review
- a metrics library normalized by exposure
- training recipes and shared compute
- a protocol for testing a vendor's system
Each unit owns
- its scenarios and edge cases
- site maps and real incident data
- acceptance thresholds
- feedback from operations
The tools are within reach
Isaac Lab, LeRobot and pretrained robot models like NVIDIA GR00T N1.7 run on one workstation. Read each license first: both arm models we used ship non-commercial terms.
Training is cheap
2 billion steps took 2 hours. Knowing whether the result was real took days.
Vendors arrive first
You'll be asked to approve, buy or answer for systems you didn't build.
Skills transfer
Agent and LLM teams already have half of it: rollouts, evals, reward design.
One workstation
Install a simulator, train a toy task, run a pretrained robot.
One real scenario
Rebuild a situation from your own operations, with a scripted baseline.
Evaluation first
Controls, held-out seeds, video review, per-exposure metrics. Before any training.
Train, score, share
Train a policy or test a vendor's. Publish the full table. Hand the kit to a second team.