Deep-Tech Decoded: #9 Robot Training 101 Part 1: How Machines Learn to Move
Over the past few weeks, a surprising number of classmates have asked me for a “robot training 101.” Maybe that’s the unofficial sign that a frontier-tech field is heating up: first the researchers get excited, then the founders, then MBAs start asking how it works 😉
What I also noticed, though, was that explaining robot training out loud is a different muscle from understanding it or writing about it. Most of my conversations so far have been with Stanford lab researchers, where the discussion can quickly get technical. But translating the same ideas for smart non-engineers forced me to ask a harder question: what is the simplest version of this field that is still accurate?
Since I love talking about this space anyway, I thought I’d write the version I wish existed: friendly enough for a smart non-engineer, but precise enough to make the core ideas stick.
A chatbot can be wrong and remain safely inside the screen.
A robot cannot.
When a language model makes a mistake, it produces a bad sentence. When a robot makes a mistake, it may drop a glass, pinch a finger, scrape a countertop, block a hallway, or push too hard against an object it does not understand.
That is why robot learning is not simply “AI with arms and legs.” It is a different category of intelligence. The model is not only predicting information. It is choosing physical action.
The core idea is simple:
Robot training is the process of turning experience into a policy — a model that maps what the robot observes to what it should do next.
That sentence contains most of the field.
A robot needs to see the world. It needs to understand the task. It needs to know where its own body is. It needs to decide how to move. It needs to sense contact. It needs to recover when something slips. And it needs to do all of this safely, in a world that is messy, changing, and full of edge cases.
The first mental model is this:
Robots are not trained to know things. They are trained to do things.
From prediction to movement
Modern AI is often explained through prediction. A language model predicts the next token. An image model predicts visual patterns. A recommendation model predicts what a user may click, watch, or buy.
A robot also predicts. But its prediction becomes movement. Instead of predicting the next word, a robot policy predicts the next physical action.
That action could be tiny: move the gripper two centimeters left, rotate the wrist, close the fingers slightly, slow down, stop.
Or it could be higher-level: pick up the cup, open the drawer, walk across the room, hand the object to a person.
Useful robots need both. They need high-level understanding of the goal and low-level control of the body.
This is why a simple instruction like “put the mug in the sink” is not simple at all.
A human hears that sentence and fills in thousands of invisible assumptions: find the mug, approach it, choose a grasp point, avoid the plate, lift with enough force, navigate to the sink, place it gently, release.
A robot has to learn those assumptions from data.
The policy is the robot’s decision engine
In robot learning, the most important word is policy.
A policy is the model that decides what action to take based on the current situation.
In its simplest form:
observation → action
The observation is what the robot knows right now. It may include camera images, depth, joint positions, gripper state, force readings, tactile signals, and a language instruction.
The action is what the robot does next. It may be a motor command, a gripper command, a movement trajectory, or a short sequence of planned motions.
So if an LLM turns text context into the next word, a robot policy turns physical context into the next move.
That is the bridge from AI to robotics.
Older robots were often programmed with explicit rules. Engineers defined the motion path, the object location, the grip pattern, and the safe operating zone. This works well in structured environments, like factories, where objects are predictable and tasks repeat.
But the real world is not structured.
Homes are cluttered. Warehouses change. Restaurants are chaotic. People move unpredictably. Lighting shifts. Objects deform. A towel is not a cup. A ripe tomato is not a metal bolt.
Modern robot training tries to replace brittle hand-coded behavior with learned policies that can adapt.
The goal is not magic. The goal is to stop writing every rule by hand.
What counts as robot experience?
For a robot, experience is not just video.
It is a record of what the robot saw, what it felt, what it did, and what happened next.
A common unit is a trajectory: a time-ordered sequence of observations and actions.
Imagine a robot learning to open a drawer. One trajectory might include the camera view of the drawer, the robot’s joint positions, the gripper moving toward the handle, the force signal when contact happens, the pull command, the drawer opening, and the final success label.
An episode is one complete attempt at a task.
A demonstration is usually a successful episode shown by a human, often through teleoperation. The human controls the robot. The robot records the observations and actions. The model learns to imitate the pattern.
This is why robotics data is so different from internet data.
A robot does not just need examples of objects. It needs examples of interaction.
It needs to know what happens when it touches the world.
The four ways robots learn
Most robot-learning methods can be understood through four buckets: imitation learning, reinforcement learning, diffusion policies, and vision-language-action models.
Each teaches the robot something different.
My personal belief is that no robot can be trained meaningfully via using only one type of learning - it’s better to have a mix. Let’s dive into what each type of learning is, to understand the pros and cons and how they could complement each other.
1. Imitation learning: copy the expert
Imitation learning is the apprenticeship model of robotics.
A human demonstrates a task, and the robot learns to copy the pattern.
For example, a person might teleoperate a robot arm to pick up a sponge, wipe a table, and place the sponge back down. The system records the images, robot positions, gripper commands, and outcome. After many examples, the policy learns what actions tend to follow what situations.
This is practical because humans already know how to do useful tasks. Instead of asking a robot to discover everything from scratch, we show it what good behavior looks like.
The simplest version is behavior cloning: when the observation looks like this, take this action.
But imitation has a weakness. It teaches the robot what success looks like. It does not always teach the robot what to do after failure.
A human demo may show the perfect way to pick up a cup. But what if the robot nudges the cup? What if the cup rotates? What if the gripper closes too early? What if the object slips?
Small errors push the robot into situations it may never have seen in training.
That is why perfect demonstrations are not enough. Robots also need corrections, failures, and recovery examples.
A robot trained only on clean demos may look impressive in a video and fragile in the real world.
Companies / labs to know: AgiBot, Tesla Optimus, Mobile ALOHA / Stanford-DeepMind research. AgiBot has been reported to combine teleoperation with reinforcement learning for manufacturing tasks, Tesla has shifted Optimus training toward human video collection, and Mobile ALOHA is a well-known imitation-learning reference point for bimanual mobile manipulation.
2. Reinforcement learning: practice with consequences
Reinforcement learning trains a robot through trial and error.
The robot tries actions. It receives rewards for good outcomes and penalties for bad ones. Over time, it learns which behaviors lead to success.
This is powerful because the robot can discover strategies that humans did not explicitly demonstrate. It can improve through practice.
But physical practice is expensive.
If a simulated robot falls 10,000 times, nothing breaks. If a real humanoid falls 10,000 times, the lab has a problem.
That is why reinforcement learning is often used in simulation, in controlled settings, or as a fine-tuning layer after imitation learning. It is especially useful for skills like locomotion, where balance, recovery, and adaptation matter.
A simple way to think about it:
Imitation learning gives the robot a first draft. Reinforcement learning helps it practice.
But practice in robotics has a cost. Robots move slowly. Hardware wears down. Unsafe behavior is unacceptable. Real-world data is precious because it is expensive to collect.
This is one reason robotics has not scaled like language models. The internet already had text. It did not already have billions of clean robot trajectories.
Companies / labs to know: Boston Dynamics, Google DeepMind, Skild AI. Boston Dynamics has publicly described integrating reinforcement learning into Spot’s locomotion, while Skild AI describes simulation-heavy training for robots across diverse bodies and conditions.
3. Diffusion policy: generate the motion, not just the next move
Diffusion policy is one of the more important recent ideas in robot learning.
The intuition comes from image generation.
An image diffusion model starts with noise and gradually turns it into a coherent image. A diffusion policy does something similar, but for action.
Instead of generating a picture, it generates a sequence of robot movements.
This matters because many physical tasks have more than one correct solution. A robot can pick up a cup from the left, from the right, or from above. It can fold a cloth in different ways. It can move around an obstacle through several valid paths.
Some models struggle when the same situation has many possible next actions. They average the options and produce a bad middle path.
A diffusion policy can represent multiple plausible futures. It can generate a smooth action sequence that fits the scene.
The key idea:
A diffusion policy does not just choose the next tiny move. It sketches the next few seconds of motion.
That is useful for manipulation, where timing and coordination matter.
Picking up a soft object is not one action. It is a sequence: approach, align, touch, adjust, close, lift, stabilize.
The quality of the motion matters as much as the goal.
Companies / labs to know: Toyota Research Institute, Columbia / Diffusion Policy research, Physical Intelligence. TRI explicitly announced a generative AI approach based on Diffusion Policy to teach robots dexterous skills; the original Diffusion Policy research framed robot behavior as conditional denoising over action sequences; Physical Intelligence’s π0 uses a related generative policy direction for generalist robot control.
4. Vision-language-action models: connect meaning to movement
The next frontier is the vision-language-action model, often called a VLA.
A VLA connects three things:
Vision: what the robot sees.
Language: what the human asks.
Action: what the robot does.
This is where robotics begins to look more like foundation-model AI.
A traditional robot might be trained for one narrow task: pick up the red block. A VLA-style robot aims to understand broader instructions, such as “move the empty cup next to the coffee machine” or “put the toy back where it belongs.”
That requires more than motor control. It requires meaning.
The robot must know what a cup is, what “empty” means, what “next to” means, and what a coffee machine looks like. Then it must translate that understanding into motion.
This is why large AI models matter for robotics. Internet-scale image and language training can give robots useful world knowledge.
But world knowledge is not the same as physical skill.
A model may understand the word “drawer.” That does not mean it knows how hard to pull one.
Robots need both semantic intelligence and embodied control.
That is the central challenge of physical AI: connecting meaning to movement.
Companies / labs to know: Physical Intelligence, Google DeepMind, Figure AI, Covariant. Physical Intelligence’s π0 is described as a general-purpose robot foundation model trained to follow text instructions; Figure’s Helix is positioned as a humanoid VLA model; Covariant’s RFM-1 combines internet data with real-world robot interaction data for warehouse robotics.
Why robot learning is hard
Robotics is hard because the world pushes back.
In software, the environment is mostly symbolic. In robotics, the environment has friction, weight, texture, lighting, occlusion, gravity, and humans.
A robot can see a shirt, but fabric deforms. It can see a tomato, but pressure matters. It can see a door handle, but the hinge may be stiff. It can see a person, but that person may suddenly move.
Four problems show up again and again.
First, contact is difficult. Many useful tasks require touching, pushing, pulling, sliding, twisting, or grasping. Once contact happens, vision alone may not be enough. The robot may need force or tactile feedback.
Second, small errors compound. If a gripper is slightly misaligned, the object may slip. If the object slips, the next observation changes. If the model has never seen that state, it may make things worse.
Third, generalization is hard. A policy trained in one lab may fail in another room. A robot that picks up one mug may fail on a transparent glass, a heavy ceramic cup, or a mug with an unusual handle.
Fourth, long-horizon tasks are fragile. A robot may succeed at grasping but fail at the fifth step of cleaning a table. The more steps a task has, the more chances there are to drift.
This is why robot demos can be misleading.
A single successful run proves the robot succeeded once, under those conditions.
The real question is different:
How often does it succeed, across how much variation, with how little human help, and how safely?
Evaluation is not just accuracy
In most AI products, evaluation can happen on test sets, user studies, or online metrics.
Robotics needs those too. But it also needs physical proof.
A useful robot policy is usually tested in layers.
Offline evaluation asks whether the model predicts reasonable actions on held-out data.
Simulation asks whether it works across many synthetic variations.
Lab testing asks whether it works on the real robot in a controlled setting.
Pilot deployment asks whether it works in a real environment with limited scope.
Full deployment asks whether it remains reliable over time.
The important metrics are physical operating metrics:
Task success rate.
Time to completion.
Collision rate.
Human intervention rate.
Recovery rate.
Performance on new objects.
Performance in new environments.
Safety under ambiguous instructions.
For PMs and investors, the key point is:
A robot is not valuable because it can do a task once. It is valuable when it can do the task repeatedly, safely, and economically.
That is why robotics progress can look slower than software AI progress.
The demo bar is low. The deployment bar is high.
Safety is part of training
Safety in robotics is not a final checklist before launch.
It has to be part of the training loop.
A robot must learn how to act, but also when not to act.
If a human hand enters the workspace, the robot should slow down or stop. If the instruction is unsafe, it should refuse. If confidence is low, it should ask for help. If force exceeds a threshold, it should back off.
Safety has many layers.
Mechanical safety limits force, speed, torque, and range of motion.
Control safety prevents unstable movements.
Perception safety detects people, obstacles, fragile objects, and unexpected changes.
Policy safety prevents the learned model from selecting dangerous actions.
Semantic safety helps the robot understand that some instructions should not be followed.
Human override allows a person to stop, correct, or take over.
This is another reason data matters. Safe robots need more than successful demonstrations. They need near misses, human interventions, failed grasps, ambiguous scenes, unsafe commands, and recovery behavior.
In robotics, failure data is not a footnote.
It is training material.
The training loop is the product
The most important robotics companies will not just build better machines.
They will build better learning loops.
The loop looks like this:
Define the task.
Collect demonstrations.
Train a policy.
Test it in simulation and the lab.
Deploy in a limited environment.
Capture successes, failures, interventions, and edge cases.
Improve the policy.
Repeat.
This loop is the real asset.
A company with robots in the field can collect data that a research lab cannot. A company with better teleoperation can generate demonstrations faster. A company with better simulation can test rare scenarios more cheaply. A company with better evaluation can improve without breaking old skills. A company with stronger safety infrastructure can deploy earlier and learn faster.
That is how robot data becomes a moat.
But not all data is equal.
Ten thousand near-identical pick-and-place demos may be less valuable than a smaller set of diverse episodes across objects, environments, failures, and recovery cases.
The better question is not:
How much data does this company have?
It is:
Does this company have a loop that makes the robot measurably more useful over time?
That is the commercial heart of robot training.
The simple version
Robot training can sound intimidating because the field has so many terms: policies, trajectories, imitation learning, reinforcement learning, diffusion, VLAs, sim-to-real, embodiment, teleoperation.
But the basic loop is simple.
A robot observes the world.
A policy chooses an action.
The robot moves.
The world changes.
The result becomes new data.
The policy improves.
That is robot training.
The hard part is that the loop happens in the physical world, where mistakes are costly and every environment is slightly different.
This is why physical AI will not be won by the best demo alone. It will be won by the best learning system: the one that can turn messy embodied experience into reliable behavior.
The next robotics race will depend on better models, yes. But the deeper advantage will come from better data, better evaluation, better safety, and better feedback loops.
Software intelligence learned to speak by absorbing the internet.
Physical intelligence will learn differently.
It will learn by seeing, touching, failing, recovering, and trying again - until movement becomes reliable enough to leave the lab.