The motion may be correct. But the consequence is still unknown.

The cup might lift cleanly. It might slide away. It might tip over. It might spill. It might be heavier than it looks. It might be stuck to a coaster. The robot may touch the cup, but the important question comes after contact:

What happens next?

That is the gap between motion and intelligence.

A robot does not just need to know what to do. It needs to know what its action will do to the world.

That is where world models come in.

The difference between a robot that follows a motion and a robot that behaves intelligently is prediction. A world model is the layer that lets a robot ask:

What happens next if I do this?

From policy and data to prediction

In Part 1, we talked about policies: how robots choose actions. A policy maps what the robot observes to what it should do next. If the robot sees a cup, the policy may tell it to move the gripper toward the handle.

In Part 2, we talked about data: what robots need to learn from. Robot data is not just video. It is interaction data: what the robot saw, what it did, what it felt, and what happened next.

Now we move to the next layer: prediction. If the policy says, “move this way,” the world model asks, “what will happen if I move this way?”

A policy chooses. A world model predicts.

A short history: from mental models to game dreams to physical prediction

World models are not a brand-new idea.

The deeper root is the old idea that intelligent systems use internal models of reality to reason before acting. Kenneth Craik is often credited with formalizing this idea in 1943: the mind can construct “small-scale models” of reality to anticipate events before they occur.

The modern AI version became famous through David Ha and Jürgen Schmidhuber’s 2018 paper World Models. Their agent learned a compressed representation of game-like environments such as CarRacing and VizDoom, then used that learned model for control.

That game setting matters. Games are controlled worlds. The rules are hidden, but stable. If an agent can learn the dynamics of the game, it can plan inside its learned version of that world.

Robotics makes the idea harder and more useful. A physical robot cannot cheaply crash, drop, fall, or break things millions of times. DayDreamer applied Dreamer-style world models directly to real robots, including a quadruped, robot arms, and a wheeled robot, to learn from physical interaction with less dependence on simulators.

Today, the term is also used more broadly. NVIDIA describes Cosmos as a world foundation model platform for physical AI and robotics workflows. Google DeepMind describes Genie 3 as a general-purpose world model for generating interactive environments. Wayve describes GAIA-2 as a controllable video-generative world model for autonomous driving.

So the arc is not just “games to robots.” It is:

mental models → game dreams → physical prediction → generated worlds

What is a world model?

A world model is the robot’s internal predictor of what might happen next.

It does not need to perfectly simulate the universe. It does not need to model every molecule, reflection, or microscopic contact point.

It only needs to be useful.

If the robot pushes a cup, the world model predicts whether the cup may slide, tip, or stay still.

If the robot reaches around a chair, it predicts whether the arm may collide.

If the robot places a plate near the table edge, it predicts whether the placement is stable.

A world model is not the world. It is a useful guess about how the world changes.

This is why world models are so appealing in robotics. They give the robot a way to rehearse possible futures before acting in the physical world.

Policy vs. world model

The easiest way to understand a world model is to contrast it with a policy.

A policy might output: pull the drawer.

A world model predicts: the drawer may open, jam, resist, or collide with the object in front of it.

A policy is the action chooser. A world model is the consequence predictor.

The best systems may use both. The policy proposes actions. The world model helps evaluate what those actions might cause. Then the robot can choose a safer, more useful, or more reliable action.

This is robot imagination - not imagination in the poetic sense, but in the practical sense of testing futures before touching the world.

Why prediction is harder in robotics

Prediction is hard in every domain. In robotics, it is physically hard.

The world pushes back.

A cup behaves differently depending on its weight, surface friction, grip angle, and whether it is full.

A towel can fold, bunch, stretch, wrinkle, or slip.

A cable can bend, tangle, snag, or resist being pulled.

A humanoid footstep can stabilize the body or make it fall.

A person can unexpectedly move into the robot’s path.

This is different from predicting the next word in a sentence. The robot’s action changes the world, and the changed world becomes the next problem.

Small errors compound. A slightly bad grasp changes the object pose. The changed object pose makes the next action harder. The next action creates a new state the robot may not have seen before.

Physical intelligence is consequence-aware intelligence.

The robot does not just need to identify a cup. It needs to predict the future of the cup under action.

A practical map of world models

I don’t think there is one universally accepted taxonomy of world models. Researchers and companies categorize them by architecture, modality, training objective, or use case.

But as an ex consultant I always have the urge to structure things, so I tried to make a practical taxonomy here - not a canonical industry standard, but a reader-friendly map to separate ideas that often get collapsed into the same phrase.

The important point is not the taxonomy itself.

The important point is that “world model” is less a single architecture than a job description:

Predict how the world changes.

Some world models predict in a compact hidden space. Some predict videos. Some represent objects more explicitly. Some generate environments.

They are different ways of answering the same question: what might happen next?

What must a robot world model learn?

A robot world model needs to learn more than appearance.

It needs to learn the ingredients of physical consequence.

Objects: What things are, where they are, and how they can move
Physics: Sliding, falling, rolling, balance, resistance
Contact: What happens when the robot touches, pushes, grasps, or pulls
Time: How small actions compound into future states
Agents: How humans, pets, or other robots may move nearby
Uncertainty: When the robot should be unsure and act cautiously

Contact is especially hard.

Before contact, vision can do a lot. The robot can see the cup, estimate its position, and plan a path.

After contact, the world becomes less predictable. The cup may slip. The gripper may press too hard. The surface may be sticky. The object may deform. A tiny difference in angle can change the outcome.

The world is not just a scene. It is a set of possible futures.

From experience to prediction

This is where world models connect back to robot data.

Article 2 was about the ingredients: the data stack. What signal was captured? Which body produced it? What task was attempted? How was the data collected? How much time did it cover?

A world model uses that data differently.

It does not just store what happened. It learns patterns between action and consequence.

A static image can teach what a drawer looks like.

A robot episode can teach: reach toward handle, touch handle, pull, drawer opens.

A failure trace can teach: pull from the wrong angle, drawer jams.

A deployment log can teach: the same action sometimes succeeds and sometimes fails depending on object weight, friction, prior state, lighting, or human interruption.

Robot data is what happened. A world model is what the robot thinks might happen next.

A few robot-relevant world models worth knowing

The robotics industry is not using one single “world model.” The field is still early, and different teams use the phrase in different ways.

A more useful way to read the market is to look at where world models are showing up in robot learning workflows. I’ll call out some world models that I feel like are being actively used / discussed.

DayDreamer is one of the cleanest robotics examples. It applied Dreamer-style world models directly to physical robots, including robot arms, a quadruped, and a wheeled robot. The important idea is not that every robot company uses DayDreamer specifically. It is that a robot can learn a predictive model from real-world interaction, then use that model to improve behavior with fewer physical trials.

NVIDIA Cosmos represents the “world foundation model” direction for physical AI. NVIDIA positions Cosmos as a platform for building customized world models for robotics and autonomous vehicles, including predictive video worlds, synthetic data, edge cases, and robot-centric simulation. This is less about one robot policy and more about infrastructure: giving robotics teams a way to generate, simulate, and reason about physical scenarios before deploying in the real world.

Ctrl-World is a newer research example (2026) that gets closer to the problem robot companies actually care about: evaluating and improving generalist robot policies without running endless real-world rollouts. It is a controllable, multi-view world model for robot manipulation, trained on DROID trajectories, and designed to let policies roll out in “imagination space.” The paper reports that it can rank policy performance without real-world rollouts and improve policy success through synthetic successful trajectories.

Wayve GAIA-2 is not a household-robot model, but it is useful as an autonomous-driving example of where world models are commercially heading. Wayve describes GAIA-2 as a controllable video-generative world model for driving, using video, text, and action inputs to generate possible driving futures. Driving is a more constrained robotics domain than general-purpose manipulation, but the underlying logic is similar: predict how the world may evolve under actions.

The takeaway is simple: in robotics, “world model” is becoming less of a single model category and more of a prediction layer across the training stack.

Some world models help robots practice in imagination.
Some help generate synthetic scenarios.
Some help evaluate policies before physical rollout.
Some are merging into broader robot foundation models.

Not every important robot model is a world model. But many frontier systems are circling the same problem:

How do we connect physical understanding, prediction, and action?

Why world models could change the robotics race

World models matter because real-world trial and error is expensive.

A robot that can predict consequences can test more options before acting.

It can imagine several grasps before touching the object.

It can predict whether a package will slide before pushing it.

It can estimate whether a humanoid step is stable before shifting weight.

It can notice that opening a cabinet might collide with something nearby.

This can improve planning, safety, recovery, sample efficiency, and generalization.

But world models are not magic.

A bad world model can be worse than no world model if the robot trusts it too much. If the model predicts that a glass will stay stable but the glass tips, the robot still fails. If the model underestimates human movement, safety can break. If the model was trained on clean lab data, it may not predict messy homes or factories.

The goal is not to simulate the whole universe.

The goal is to predict enough of the next few seconds to act better.

For robotics companies, that could become a major advantage. The best systems will not only collect embodied data. They will turn that data into better predictions, better evaluations, safer actions, and faster learning loops.

Better prediction can turn the same experience into better behavior.

Consequence-aware robots

A robot does not need perfect physics for every object in a kitchen.

It needs enough prediction to avoid breaking the glass, spilling the drink, crushing the fruit, tangling the cable, or trapping itself in a bad state.

That is why world models matter.

They are not just another technical module. They are a way to make robot intelligence less reactive and more consequence-aware.

A policy tells the robot what to do.

A world model helps it understand what the world might do back.

The robot that wins may not be the one that moves fastest.

It may be the one that best understands what its movement will cause.

This time, I’ll talk about what must come before training - the Data.

A robot-training dataset can be tactile, egocentric, long-horizon, human-generated, and kitchen-task data at the same time.

That sentence is why robot data gets confusing.

When people talk about the “types of data” robots need, the conversation often becomes a flat list: vision data, tactile data, teleoperation data, simulation data, egocentric video, humanoid data, long-horizon data.

But those are not the same kind of category.

“Tactile” is a modality.
“Egocentric” is a viewpoint or collection method.
“Humanoid” is an embodiment.
“Kitchen task” is a task domain.
“Long-horizon” is a temporal structure.

A single dataset can be all of them at once.

That is the first shift: robot data is not a pile. It is a stack.

And the robotics data problem is not just a scale problem. It is a composition problem.

Robots do not simply need more data. They need the right mix of embodied experience: what was sensed, whose body produced it, what task was being performed, how the data was collected, and how much time the episode covered.

Text models needed the internet. Robots need something harder: a structured record of bodies acting in the world.

Robot data is interaction data

Internet data mostly records what humans wrote, said, saw, or uploaded.

Robot data records something different: an embodied system acting in the world and observing what happened next.

An image of a cup can teach a model what a cup looks like. A video of a person picking up a cup can teach intent and procedure. But a robot episode teaches more: where the gripper moved, when it touched the cup, how much force it applied, whether the cup slipped, how the robot corrected, and whether the task succeeded.

That is the core distinction:

Robot data is not just observation data. It is interaction data.

A robot does not only need to recognize a drawer. It needs to know how to reach the handle, how much resistance to expect, what to do if the drawer sticks, and how to stop if a human hand enters the workspace.

This is why robotics cannot simply copy the data playbook of language or vision models. The physical world does not just sit there waiting to be classified. It pushes back.

The five-axis data stack

A clean way to categorize robot-training data is to ask five questions.

These are not five separate buckets of data. They are five labels that can describe the same episode.

For example:

Egocentric RGB video plus hand-pose data, generated by a human body, during a kitchen task, collected through wearable capture, over a long-horizon sequence.

Or:

RGB, depth, proprioception, and action data, generated by a robot arm, during tabletop manipulation, collected through teleoperation, over a short task episode.

This is why a MECE taxonomy matters. Without it, “tactile data” and “long-horizon data” sound like competing options. They are not. One describes the signal. The other describes the time scale.

The stack makes the conversation cleaner.

1. Modality: what signal is captured?

A modality is a channel of experience.

For robots, common modalities include RGB video, depth, proprioception, force, torque, tactile pressure, language, and action commands.

Each modality teaches something different.

Vision tells the robot what the world looks like. Depth helps it understand shape and distance. Proprioception tells the robot where its own body is. Force and torque reveal resistance. Tactile data captures pressure, slip, texture, and contact. Language tells the robot the goal. Action data records what movement was actually taken.

The key line is simple:

Vision tells the robot what the world looks like. Touch and force tell it what happens when the world pushes back.

This matters most in contact-rich tasks.

A camera may see that a tomato is red and round. It may not know whether the tomato is firm, slippery, bruised, or about to burst under pressure. A camera may see a cable. It may not know how the cable bends, tangles, or resists being pulled.

The more a task depends on contact, the less sufficient vision alone becomes.

This is why tactile and force data are so strategically interesting. They add a missing layer of physical feedback. They help robots move from seeing objects to handling them.

2. Embodiment: whose body generated the data?

In robotics, the body is part of the dataset.

A human hand, a two-finger gripper, a robot arm, a quadruped, a bimanual robot, and a humanoid do not experience the same world. They have different joints, sensors, reach, balance, strength, and action spaces.

This is why human video is useful but incomplete.

A video of a person folding laundry can teach procedure: pick up the towel, find the corners, fold, align, stack. But the human hand has soft skin, many degrees of freedom, and rich tactile sensing. A robot gripper may not have those capabilities. The task is the same. The body is not.

This is also why cross-embodiment robot data matters. Open X-Embodiment contains more than one million real robot trajectories across 22 robot embodiments, from single robot arms to bimanual robots and quadrupeds. Its value is not only scale; it is the attempt to pool experience across different bodies. (Robotics Transformer X)

The lesson is subtle but important:

A dataset does not just describe a task. It describes a task performed by a particular body.

A humanoid learning from robot-arm data may gain useful priors. But it still has to translate that experience into a different body, with different constraints.

3. Task domain: what behavior is being learned?

Not all robot data teaches equally useful behavior.

A million examples of pick-and-place do not automatically teach a robot to clean a kitchen.

Task domain asks: what kind of behavior is inside the data?

The value of robot data depends on what behavior it teaches.

This is why task diversity has become so important. DROID, a large in-the-wild robot manipulation dataset, contains 76,000 demonstration trajectories, about 350 hours of interaction data, collected across 564 scenes and 86 tasks. The point is not only to collect more robot data, but to collect it across more varied environments and tasks so policies generalize better. (DROID Dataset)

For PMs and investors, the question should not be: how many episodes does the company have?

The better question is:

What behaviors do those episodes actually cover?

A robot company with 500,000 narrow lab demos may be less interesting than one with fewer but richer episodes across real-world variation, failure, and recovery.

4. Collection method: how was the data produced?

The way data is collected determines the learning signal.

Teleoperation gives robot-specific demonstrations. A human controls the robot, and the system records observations and actions. This is valuable because the data is already in the robot’s body and action space. But it can be expensive and slow.

Human video gives scale. The world has far more videos of humans doing tasks than robots doing tasks. Egocentric video is especially useful because it captures activity from a first-person perspective. Ego4D includes 3,670 hours of first-person video from 923 participants across 74 worldwide locations, making it a useful reference point for large-scale egocentric data.

But human video has a translation problem. It can show intent and procedure, but it does not directly provide robot motor commands.

Simulation gives cheap practice. Robots can fall, fail, and retry thousands of times without breaking hardware. It is useful for rare events, dangerous scenarios, and scalable variation. But simulation has a reality gap: what works in a physics engine may not work perfectly in the real world.

Synthetic data sits somewhere nearby. NVIDIA describes Isaac GR00T tools as generating large synthetic trajectory datasets from a small number of human demonstrations, using GR00T-Mimic and Cosmos as part of the data-generation workflow.

Deployment logs may be the most commercially valuable source. They capture the messiness of the real world: lighting changes, object variation, human interruptions, edge cases, and failure modes that no lab fully anticipated.

The collection method is not a logistics detail. It determines what the robot can learn.

5. Temporal structure: how much time does the data cover?

Robot data also differs by time scale.

Short data can teach skills. Long data teaches work.

A robot may learn to grasp a cup from short episodes. But clearing a table requires more: find all objects, decide what belongs where, sequence actions, avoid collisions, recover from mistakes, and know when the task is complete.

That is a different data problem.

The longer the horizon, the more fragile the behavior becomes. Every additional step creates another chance for error. A robot that succeeds at step one may still fail at step five.

This is why long-horizon data is so valuable. It teaches continuity.

The robot does not just need to perform isolated motions. It needs to carry intention across time.

The missing data: failure and recovery

The cleanest robot demos usually hide the most important training signal.

Failure.

A cup slips.
A drawer sticks.
A cable tangles.
A towel folds unevenly.
A person walks into the workspace.
The robot bumps the object and has to replan.

These moments are not just errors. They are data.

Perfect demonstrations teach the path. Failure data teaches resilience.

This matters because real robots will not live inside perfect trajectories. They will live inside distribution shifts: new homes, new warehouses, new lighting, new objects, new humans, new interruptions.

A robot trained only on clean demonstrations may perform beautifully until the world deviates. Then it may freeze, amplify the mistake, or need human intervention.

Robustness lives in the recovery trace.

That means valuable robot datasets should not only ask: did the task succeed?

They should ask: what went wrong, when did it go wrong, what did the robot sense, what correction was attempted, and did recovery work?

Why deployment data becomes a moat

This is where the technical discussion becomes strategic.

The most valuable robot data may come after deployment.

A lab can design tasks. A simulator can generate variation. A teleoperator can demonstrate skills. But the real world produces edge cases that are hard to imagine in advance.

A warehouse robot encounters unusual packages, damaged boxes, reflective surfaces, blocked aisles, and humans moving unpredictably.

A household robot encounters pets, children, clutter, low lighting, spilled liquids, furniture variation, and vague instructions.

An industrial robot encounters part variation, tool wear, alignment drift, safety interruptions, and unexpected resistance.

These are not side cases. They are the path to reliability.

That is why the real moat may not be the initial dataset. It may be the loop that keeps producing better data.

Deploy robots.
Capture real episodes.
Identify failures.
Retrain the policy.
Evaluate safety.
Redeploy.
Repeat.

The companies that own this loop can learn from the physical world faster than companies limited to lab demos or public datasets.

This does not mean every company with deployed robots automatically wins. The data has to be captured cleanly, labeled usefully, filtered intelligently, and connected back into training and evaluation.

But the strategic principle holds:

The real data moat is not just the dataset. It is the system that keeps turning real-world experience into better behavior.

The real robotics data race

The robotics race is often framed as a hardware race.

Who has the best humanoid?
Who has the strongest gripper?
Who has the most elegant demo?
Who has the cheapest robot arm?

Hardware matters. Models matter. But neither is enough.

The deeper race is about learning infrastructure: the ability to collect, structure, evaluate, and reuse embodied experience.

That is why robot data should not be treated as a flat list of data types. “Vision,” “tactile,” “egocentric,” “simulation,” and “long-horizon” are useful words, but they answer different questions.

A better frame is the stack:

What signal was captured?
Whose body produced it?
What task was being learned?
How was the data collected?
How much time did it cover?

Once you see the stack, the bottleneck becomes clearer.

Robots do not need data in the abstract. They need the right composition of experience: enough sensory richness to perceive and feel, enough embodiment diversity to generalize across bodies, enough task coverage to become useful, enough real-world grounding to avoid brittle behavior, and enough temporal depth to complete real jobs.

Software intelligence learned from the internet. Physical intelligence will learn from the world - but only if that world is captured as more than video.

It must be captured as embodied experience: sensed, acted, felt, failed, recovered, and repeated.

Robot data is not a pile.

It is a stack.

What I also noticed, though, was that explaining robot training out loud is a different muscle from understanding it or writing about it. Most of my conversations so far have been with Stanford lab researchers, where the discussion can quickly get technical. But translating the same ideas for smart non-engineers forced me to ask a harder question: what is the simplest version of this field that is still accurate?

Since I love talking about this space anyway, I thought I’d write the version I wish existed: friendly enough for a smart non-engineer, but precise enough to make the core ideas stick.

A chatbot can be wrong and remain safely inside the screen.

A robot cannot.

When a language model makes a mistake, it produces a bad sentence. When a robot makes a mistake, it may drop a glass, pinch a finger, scrape a countertop, block a hallway, or push too hard against an object it does not understand.

That is why robot learning is not simply “AI with arms and legs.” It is a different category of intelligence. The model is not only predicting information. It is choosing physical action.

The core idea is simple:

Robot training is the process of turning experience into a policy — a model that maps what the robot observes to what it should do next.

That sentence contains most of the field.

A robot needs to see the world. It needs to understand the task. It needs to know where its own body is. It needs to decide how to move. It needs to sense contact. It needs to recover when something slips. And it needs to do all of this safely, in a world that is messy, changing, and full of edge cases.

The first mental model is this:

Robots are not trained to know things. They are trained to do things.

From prediction to movement

Modern AI is often explained through prediction. A language model predicts the next token. An image model predicts visual patterns. A recommendation model predicts what a user may click, watch, or buy.

A robot also predicts. But its prediction becomes movement. Instead of predicting the next word, a robot policy predicts the next physical action.

That action could be tiny: move the gripper two centimeters left, rotate the wrist, close the fingers slightly, slow down, stop.

Or it could be higher-level: pick up the cup, open the drawer, walk across the room, hand the object to a person.

Useful robots need both. They need high-level understanding of the goal and low-level control of the body.

This is why a simple instruction like “put the mug in the sink” is not simple at all.

A human hears that sentence and fills in thousands of invisible assumptions: find the mug, approach it, choose a grasp point, avoid the plate, lift with enough force, navigate to the sink, place it gently, release.

A robot has to learn those assumptions from data.

The policy is the robot’s decision engine

In robot learning, the most important word is policy.

A policy is the model that decides what action to take based on the current situation.

In its simplest form:

observation → action

The observation is what the robot knows right now. It may include camera images, depth, joint positions, gripper state, force readings, tactile signals, and a language instruction.

The action is what the robot does next. It may be a motor command, a gripper command, a movement trajectory, or a short sequence of planned motions.

So if an LLM turns text context into the next word, a robot policy turns physical context into the next move.

That is the bridge from AI to robotics.

Older robots were often programmed with explicit rules. Engineers defined the motion path, the object location, the grip pattern, and the safe operating zone. This works well in structured environments, like factories, where objects are predictable and tasks repeat.

But the real world is not structured.

Homes are cluttered. Warehouses change. Restaurants are chaotic. People move unpredictably. Lighting shifts. Objects deform. A towel is not a cup. A ripe tomato is not a metal bolt.

Modern robot training tries to replace brittle hand-coded behavior with learned policies that can adapt.

The goal is not magic. The goal is to stop writing every rule by hand.

What counts as robot experience?

For a robot, experience is not just video.

It is a record of what the robot saw, what it felt, what it did, and what happened next.

A common unit is a trajectory: a time-ordered sequence of observations and actions.

Imagine a robot learning to open a drawer. One trajectory might include the camera view of the drawer, the robot’s joint positions, the gripper moving toward the handle, the force signal when contact happens, the pull command, the drawer opening, and the final success label.

An episode is one complete attempt at a task.

A demonstration is usually a successful episode shown by a human, often through teleoperation. The human controls the robot. The robot records the observations and actions. The model learns to imitate the pattern.

This is why robotics data is so different from internet data.

A robot does not just need examples of objects. It needs examples of interaction.

It needs to know what happens when it touches the world.

The four ways robots learn

Most robot-learning methods can be understood through four buckets: imitation learning, reinforcement learning, diffusion policies, and vision-language-action models.

Each teaches the robot something different.

My personal belief is that no robot can be trained meaningfully via using only one type of learning - it’s better to have a mix. Let’s dive into what each type of learning is, to understand the pros and cons and how they could complement each other.

1. Imitation learning: copy the expert

Imitation learning is the apprenticeship model of robotics.

A human demonstrates a task, and the robot learns to copy the pattern.

For example, a person might teleoperate a robot arm to pick up a sponge, wipe a table, and place the sponge back down. The system records the images, robot positions, gripper commands, and outcome. After many examples, the policy learns what actions tend to follow what situations.

This is practical because humans already know how to do useful tasks. Instead of asking a robot to discover everything from scratch, we show it what good behavior looks like.

The simplest version is behavior cloning: when the observation looks like this, take this action.

But imitation has a weakness. It teaches the robot what success looks like. It does not always teach the robot what to do after failure.

A human demo may show the perfect way to pick up a cup. But what if the robot nudges the cup? What if the cup rotates? What if the gripper closes too early? What if the object slips?

Small errors push the robot into situations it may never have seen in training.

That is why perfect demonstrations are not enough. Robots also need corrections, failures, and recovery examples.

A robot trained only on clean demos may look impressive in a video and fragile in the real world.

• Companies / labs to know: AgiBot, Tesla Optimus, Mobile ALOHA / Stanford-DeepMind research. AgiBot has been reported to combine teleoperation with reinforcement learning for manufacturing tasks, Tesla has shifted Optimus training toward human video collection, and Mobile ALOHA is a well-known imitation-learning reference point for bimanual mobile manipulation.

2. Reinforcement learning: practice with consequences

Reinforcement learning trains a robot through trial and error.

The robot tries actions. It receives rewards for good outcomes and penalties for bad ones. Over time, it learns which behaviors lead to success.

This is powerful because the robot can discover strategies that humans did not explicitly demonstrate. It can improve through practice.

But physical practice is expensive.

If a simulated robot falls 10,000 times, nothing breaks. If a real humanoid falls 10,000 times, the lab has a problem.

That is why reinforcement learning is often used in simulation, in controlled settings, or as a fine-tuning layer after imitation learning. It is especially useful for skills like locomotion, where balance, recovery, and adaptation matter.

A simple way to think about it:

Imitation learning gives the robot a first draft. Reinforcement learning helps it practice.

But practice in robotics has a cost. Robots move slowly. Hardware wears down. Unsafe behavior is unacceptable. Real-world data is precious because it is expensive to collect.

This is one reason robotics has not scaled like language models. The internet already had text. It did not already have billions of clean robot trajectories.

• Companies / labs to know: Boston Dynamics, Google DeepMind, Skild AI. Boston Dynamics has publicly described integrating reinforcement learning into Spot’s locomotion, while Skild AI describes simulation-heavy training for robots across diverse bodies and conditions.

3. Diffusion policy: generate the motion, not just the next move

Diffusion policy is one of the more important recent ideas in robot learning.

The intuition comes from image generation.

An image diffusion model starts with noise and gradually turns it into a coherent image. A diffusion policy does something similar, but for action.

Instead of generating a picture, it generates a sequence of robot movements.

This matters because many physical tasks have more than one correct solution. A robot can pick up a cup from the left, from the right, or from above. It can fold a cloth in different ways. It can move around an obstacle through several valid paths.

Some models struggle when the same situation has many possible next actions. They average the options and produce a bad middle path.

A diffusion policy can represent multiple plausible futures. It can generate a smooth action sequence that fits the scene.

The key idea:

A diffusion policy does not just choose the next tiny move. It sketches the next few seconds of motion.

That is useful for manipulation, where timing and coordination matter.

Picking up a soft object is not one action. It is a sequence: approach, align, touch, adjust, close, lift, stabilize.

The quality of the motion matters as much as the goal.

• Companies / labs to know: Toyota Research Institute, Columbia / Diffusion Policy research, Physical Intelligence. TRI explicitly announced a generative AI approach based on Diffusion Policy to teach robots dexterous skills; the original Diffusion Policy research framed robot behavior as conditional denoising over action sequences; Physical Intelligence’s π0 uses a related generative policy direction for generalist robot control.

4. Vision-language-action models: connect meaning to movement

The next frontier is the vision-language-action model, often called a VLA.

A VLA connects three things:

Vision: what the robot sees.
Language: what the human asks.
Action: what the robot does.

This is where robotics begins to look more like foundation-model AI.

A traditional robot might be trained for one narrow task: pick up the red block. A VLA-style robot aims to understand broader instructions, such as “move the empty cup next to the coffee machine” or “put the toy back where it belongs.”

That requires more than motor control. It requires meaning.

The robot must know what a cup is, what “empty” means, what “next to” means, and what a coffee machine looks like. Then it must translate that understanding into motion.

This is why large AI models matter for robotics. Internet-scale image and language training can give robots useful world knowledge.

But world knowledge is not the same as physical skill.

A model may understand the word “drawer.” That does not mean it knows how hard to pull one.

Robots need both semantic intelligence and embodied control.

That is the central challenge of physical AI: connecting meaning to movement.

• Companies / labs to know: Physical Intelligence, Google DeepMind, Figure AI, Covariant. Physical Intelligence’s π0 is described as a general-purpose robot foundation model trained to follow text instructions; Figure’s Helix is positioned as a humanoid VLA model; Covariant’s RFM-1 combines internet data with real-world robot interaction data for warehouse robotics.

Why robot learning is hard

Robotics is hard because the world pushes back.

In software, the environment is mostly symbolic. In robotics, the environment has friction, weight, texture, lighting, occlusion, gravity, and humans.

A robot can see a shirt, but fabric deforms. It can see a tomato, but pressure matters. It can see a door handle, but the hinge may be stiff. It can see a person, but that person may suddenly move.

Four problems show up again and again.

First, contact is difficult. Many useful tasks require touching, pushing, pulling, sliding, twisting, or grasping. Once contact happens, vision alone may not be enough. The robot may need force or tactile feedback.

Second, small errors compound. If a gripper is slightly misaligned, the object may slip. If the object slips, the next observation changes. If the model has never seen that state, it may make things worse.

Third, generalization is hard. A policy trained in one lab may fail in another room. A robot that picks up one mug may fail on a transparent glass, a heavy ceramic cup, or a mug with an unusual handle.

Fourth, long-horizon tasks are fragile. A robot may succeed at grasping but fail at the fifth step of cleaning a table. The more steps a task has, the more chances there are to drift.

This is why robot demos can be misleading.

A single successful run proves the robot succeeded once, under those conditions.

The real question is different:

How often does it succeed, across how much variation, with how little human help, and how safely?

Evaluation is not just accuracy

In most AI products, evaluation can happen on test sets, user studies, or online metrics.

Robotics needs those too. But it also needs physical proof.

A useful robot policy is usually tested in layers.

Offline evaluation asks whether the model predicts reasonable actions on held-out data.

Simulation asks whether it works across many synthetic variations.

Lab testing asks whether it works on the real robot in a controlled setting.

Pilot deployment asks whether it works in a real environment with limited scope.

Full deployment asks whether it remains reliable over time.

The important metrics are physical operating metrics:

Task success rate.
Time to completion.
Collision rate.
Human intervention rate.
Recovery rate.
Performance on new objects.
Performance in new environments.
Safety under ambiguous instructions.

For PMs and investors, the key point is:

A robot is not valuable because it can do a task once. It is valuable when it can do the task repeatedly, safely, and economically.

That is why robotics progress can look slower than software AI progress.

The demo bar is low. The deployment bar is high.

Safety is part of training

Safety in robotics is not a final checklist before launch.

It has to be part of the training loop.

A robot must learn how to act, but also when not to act.

If a human hand enters the workspace, the robot should slow down or stop. If the instruction is unsafe, it should refuse. If confidence is low, it should ask for help. If force exceeds a threshold, it should back off.

Safety has many layers.

Mechanical safety limits force, speed, torque, and range of motion.

Control safety prevents unstable movements.

Perception safety detects people, obstacles, fragile objects, and unexpected changes.

Policy safety prevents the learned model from selecting dangerous actions.

Semantic safety helps the robot understand that some instructions should not be followed.

Human override allows a person to stop, correct, or take over.

This is another reason data matters. Safe robots need more than successful demonstrations. They need near misses, human interventions, failed grasps, ambiguous scenes, unsafe commands, and recovery behavior.

In robotics, failure data is not a footnote.

It is training material.

The training loop is the product

The most important robotics companies will not just build better machines.

They will build better learning loops.

The loop looks like this:

Define the task.
Collect demonstrations.
Train a policy.
Test it in simulation and the lab.
Deploy in a limited environment.
Capture successes, failures, interventions, and edge cases.
Improve the policy.
Repeat.

This loop is the real asset.

A company with robots in the field can collect data that a research lab cannot. A company with better teleoperation can generate demonstrations faster. A company with better simulation can test rare scenarios more cheaply. A company with better evaluation can improve without breaking old skills. A company with stronger safety infrastructure can deploy earlier and learn faster.

That is how robot data becomes a moat.

But not all data is equal.

Ten thousand near-identical pick-and-place demos may be less valuable than a smaller set of diverse episodes across objects, environments, failures, and recovery cases.

The better question is not:

How much data does this company have?

It is:

Does this company have a loop that makes the robot measurably more useful over time?

That is the commercial heart of robot training.

The simple version

Robot training can sound intimidating because the field has so many terms: policies, trajectories, imitation learning, reinforcement learning, diffusion, VLAs, sim-to-real, embodiment, teleoperation.

But the basic loop is simple.

A robot observes the world.
A policy chooses an action.
The robot moves.
The world changes.
The result becomes new data.
The policy improves.

That is robot training.

The hard part is that the loop happens in the physical world, where mistakes are costly and every environment is slightly different.

This is why physical AI will not be won by the best demo alone. It will be won by the best learning system: the one that can turn messy embodied experience into reliable behavior.

The next robotics race will depend on better models, yes. But the deeper advantage will come from better data, better evaluation, better safety, and better feedback loops.

Software intelligence learned to speak by absorbing the internet.

Physical intelligence will learn differently.

It will learn by seeing, touching, failing, recovering, and trying again - until movement becomes reliable enough to leave the lab.

• Apple may look late to AI, but it may be playing a different game.

• Model companies are racing to build the smartest AI; Apple wants to own the doorway.

• Tim Cook built the fortress, but not the next iPhone.

• John Ternus signals Apple’s bet on AI as a device-native, ambient experience.

Everyone is asking the wrong question about Apple.

The question is usually: Can Apple catch up in AI?

It is easy to see why. OpenAI has ChatGPT. Google has Gemini. Anthropic has Claude. NVIDIA has become the symbol of the AI boom. Meanwhile, Apple’s AI rollout has felt cautious, delayed, and strangely quiet for a company that once made the future feel obvious.

But Apple has rarely won by being first to a technology.

Apple wins when it turns technology into a habit.

So the better question is not whether Apple can build the smartest AI model. The better question is:

Can Apple become the place where everyday people meet AI?

That distinction matters. One question is about raw intelligence. The other is about distribution, trust, hardware, payments, apps, and daily behavior.

Apple may be late to the AI model race. But it may still be early to the AI doorway.

Apple is weak where AI is loud, but strong where AI becomes daily

The visible AI race is happening in chat windows.

That is where Apple looks weakest. Siri, once an early symbol of consumer AI, now feels behind. ChatGPT can write, reason, code, summarize, and explain. Gemini is being pushed across Google’s products. Claude has become a favorite among knowledge workers.

Siri still often feels like a voice command tool from another era.

That is the obvious story.

The less obvious story is that most people do not choose AI by comparing benchmark scores. They choose what is already in front of them.

They do not wake up wondering which foundation model has the strongest reasoning performance this week.

They wonder: Can my phone help me get this done?

That is Apple’s opening.

Apple still controls some of the most valuable consumer surfaces in technology: the iPhone in your hand, the Mac on your desk, the AirPods in your ears, the Watch on your wrist, the App Store where apps are discovered, Apple Pay where transactions happen, and iOS where permissions are granted.

OpenAI, Google, and Anthropic may own powerful AI brains. But Apple owns many of the places those brains need to live.

A simple analogy: the model companies are building better chefs. Apple owns the hotel - the lobby, the rooms, the concierge desk, and the payment system.

The chef matters. But the guest relationship often belongs to the hotel.

Apple’s real strategy: own the switchboard

Apple’s AI strategy is probably not to out-OpenAI OpenAI.

It is to make Siri, iOS, and Apple devices the routing layer for AI.

Imagine asking Siri to plan dinner with a friend. That sounds simple. But it requires multiple small jobs: read the message, check your calendar, understand location, compare restaurants, maybe book a table, and draft a reply.

No single “AI answer” is enough. The system needs to move across apps, data, permissions, and actions.

That is where Apple wants to sit.

The user says: “Plan dinner with Sarah next Thursday.”

Behind the scenes, Apple decides what should handle each part: an Apple model, Gemini, ChatGPT, Claude, Maps, Calendar, Messages, or a restaurant app.

The user does not care which model did the work. The user only cares that it worked.

That is the switchboard strategy. Apple does not need to own every AI brain. It needs to decide which brain gets called, when, and inside what experience.

This is why Apple’s partnerships with external AI models should not be read only as weakness. They may also be Apple’s classic “buy the ingredient, own the experience” move.

Apple did not manufacture every component in the iPhone. It controlled the product. Apple did not invent every technology inside the Mac. It controlled the experience.

AI may follow the same logic. Apple wants models to become suppliers. Apple wants to remain the interface.

Why this could work: Apple owns the stack

Apple’s advantage is not one thing. It is the stack.

A stack simply means the layers that make a product work. In Apple’s case, that stack includes chips, devices, operating systems, apps, services, payments, identity, and retail distribution.

Most companies own one or two layers.

Apple owns many.

That matters more in AI than it did in the app era.

AI is not just another app category. The more useful AI becomes, the more access it needs. It needs your messages to summarize conversations. Your calendar to schedule things. Your photos to find memories. Your files to retrieve documents. Your location to make recommendations. Your payment tools to complete transactions.

That is powerful. It is also sensitive.

A personal AI agent is only useful if it can touch your life. But the more it touches your life, the more trust matters.

This is where Apple has a real advantage. For years, Apple has trained users to see it as the privacy-first consumer technology company. That positioning becomes more valuable when AI shifts from answering questions to taking actions.

Because in the agent era, trust is not branding.

Trust is infrastructure.

The second layer is hardware.

AI sounds like software, but it is deeply physical. It needs chips, memory, batteries, sensors, microphones, cameras, and thermal management. If AI runs on your phone, it must feel instant, private, and efficient enough not to destroy battery life.

This is why Apple Silicon matters.

Apple’s chip strategy is not just a performance story. It gives Apple control over how intelligence runs across its devices. Some tasks can happen on-device. Some can go to Apple’s private cloud. Some can go to external models. The user sees one experience. Apple orchestrates the work behind the curtain.

Think of Apple Silicon as the nervous system.

The AI may be the brain, but the nervous system decides how quickly, privately, and smoothly signals move through the body.

Tim Cook built the fortress. He did not build the next iPhone.

This is where the CEO transition becomes strategically important.

Apple announced that Tim Cook will become executive chairman and John Ternus, Apple’s senior vice president of Hardware Engineering, will become CEO effective September 1, 2026. Apple described the transition as the result of a long-term succession process approved unanimously by its board.

Cook’s record is extraordinary.

He turned Apple into one of the most valuable companies in the world. He built a world-class supply chain. He expanded services. He scaled Apple Watch and AirPods. He made Apple less dependent on one heroic product launch.

But one critique stayed with him: Cook did not create the next iPhone.

Apple Car was canceled after years of effort. Vision Pro is technically impressive, but it has not become a mass-market platform. Its problem is not only price. It is clarity. Most consumers still do not have an instant answer to: Why do I need this every day?

That is the difference between a beautiful technology and a killer product.

The iPhone did not require a long explanation. It collapsed the phone, iPod, browser, camera, and app platform into one object people immediately understood.

Vision Pro did not.

So Cook’s legacy has a paradox. He may be one of the most successful operators in corporate history, yet Apple enters the AI era with an unanswered product question:

What is the next personal computer?

Apple II made computing personal at home.
The Mac made it graphical.
The iPhone made it mobile.
The next version may be agentic.

An agent is not just software that answers. It is software that acts.

Not “here is the answer.”

More like: “I handled it.”

That is the product Apple is now chasing.

Why a hardware CEO makes sense in an AI era

At first, choosing a hardware leader to run Apple in the AI era sounds backward.

Shouldn’t the next CEO be an AI researcher? A cloud executive? A software platform leader?

Maybe - if Apple were Google or OpenAI.

But Apple’s deepest skill has never been raw invention. It is translation.

Apple turns complex technology into objects normal people understand.

That makes John Ternus’s background more meaningful. He is not just a “hardware person.” He represents a bet that the next AI platform will not live only in a chatbot. It will live in devices.

Maybe smart glasses let AI see what you see.
Maybe AirPods become an always-available assistant.
Maybe Apple Watch becomes the health and context layer.
Maybe the home gets a new AI device.
Maybe the iPhone itself becomes less like an app launcher and more like an agent hub.

The common thread is simple:

AI will not stay inside a text box. It will move into the devices around us.

If that is true, a hardware-centric CEO is not a retreat into Apple’s past.

It is a bet on where AI goes next.

The market implication: model companies become ingredients

If Apple succeeds, the AI market may not evolve the way people expect.

Right now, model companies are the stars. They compete on intelligence, speed, cost, and brand.

But consumer technology often rewards the company that owns the interface.

Most people do not know which payment processor powers a checkout page. They know the store.

Most people do not know which supplier made a phone component. They know the device.

Most people may not know which model answered a question if the answer appears naturally inside Siri, Messages, Mail, Photos, or Calendar.

That is the opportunity.

Apple can let model companies fight the expensive intelligence war while it controls the consumer relationship.

This does not mean models are unimportant. Bad AI will break the experience. But if multiple models become good enough for many consumer tasks, distribution becomes more powerful.

That is when Apple becomes dangerous.

Model companies need users. Apple has users.
Model companies need trusted surfaces. Apple has trusted surfaces.
Model companies need monetization. Apple has payments.
Model companies need context. Apple has the device ecosystem where context lives.

This is why Apple can look late and still matter. 🍎💪

What to watch next

The first signal is Siri.

Do not watch whether Siri gets a prettier animation. Watch whether it can complete multi-step tasks across apps. Can it find the right email, summarize the context, make a decision, and take action with permission?

The second signal is model choice.

If Apple lets users access multiple AI models through Siri or system settings, Apple becomes the switchboard. Model competition becomes Apple leverage.

The third signal is on-device AI.

Listen for Apple talking about privacy, local inference, neural engines, Apple Silicon, and private cloud execution. That language means Apple is not just adding AI features. It is building AI into the architecture of the product.

The fourth signal is new hardware.

Smart glasses, AI-enabled AirPods, home robots, foldables, or new ambient devices would suggest Apple believes the next interface is not a chat window.

The fifth signal is developer access.

If developers can plug their apps into Siri and Apple Intelligence in a useful way, Apple’s AI strategy becomes a platform strategy.

That is when the story gets much bigger.

The door still has to work

There is one major caveat.

Apple cannot win the AI doorway if the doorway is broken.

If Siri remains unreliable, users will go directly to ChatGPT, Gemini, or Claude. If Apple moves too slowly, habits may form elsewhere. If regulation weakens Apple’s control over defaults and App Store economics, its gatekeeper power shrinks. If new hardware repeats the Vision Pro problem - technically beautiful, but not obviously necessary - the strategy stalls.

Apple also has a cultural challenge.

AI products improve through messy iteration. Apple prefers polish. The agent era may reward companies willing to ship imperfect tools, learn quickly, and tolerate public mistakes.

Apple has to move faster without breaking the trust that makes its strategy possible.

That is hard.

But Apple has one advantage few companies have: patience backed by distribution.

Tim Cook built the fortress.

John Ternus now has to answer the question Cook never fully answered: what is the next product that makes Apple feel inevitable again?

The answer may not look like another iPhone.

It may look like the iPhone, AirPods, Watch, Mac, and home quietly becoming one personal agent.

Not a chatbot you visit.

A layer of intelligence that follows you.

Apple does not need to build the smartest brain in AI.

It needs to make sure that when AI enters everyday life, it enters through Apple’s door.

During my consulting years, I worked closely with traditional semiconductor companies - IDMs, foundries, system players. The industry was built on specialization. Design here. Manufacturing there. Integration somewhere else.

And for a long time, I assumed that was permanent. The capital intensity was too high. The technical barriers were too deep. No single player could realistically control the entire value chain. So I used to think this kind of integration was unrealistic.

But this is exactly what makes this moment so striking to me. Why is the same person suddenly talking about chips, datacenters, and space… in one strategy?

Over the past year, Elon Musk has been floating an idea called Terafab - a proposed ~$20–25 billion semiconductor facility, with ambitions ranging from ~100,000 wafers per month initially to far larger long-term capacity, and targeting advanced AI chips.

But this is not what it looks like. This is not really about replacing TSMC.

To understand what’s really happening, we need to zoom out - from chips, to the entire system that produces AI.

1. Decode the Basics: Build the Mental Model

For decades, the semiconductor world was modular. Designers designed. Foundries manufactured. Cloud providers deployed.

That worked - until AI scaled fast enough to break the seams between them. Today, AI is constrained by four things at once: chip supply, power, cooling, and system integration. And increasingly, the hardest problems sit at the boundaries. Not inside the chip - but between chips and memory, between racks, between systems.

This is where Terafab is better understood not as a traditional fab, but as a step toward a more integrated compute system.

2. What Is Actually Happening

Tesla’s trajectory is moving from designing chips toward selectively controlling parts of the supply chain that matter most. Not necessarily to replace existing foundries - but to reduce dependency where it becomes strategically risky.

At the same time, the definition of performance is changing. Modern AI systems depend heavily on how tightly logic chips are connected to memory (like high-bandwidth memory) and how efficiently they are packaged together. In many cases, advanced packaging (not the chip itself) is becoming the bottleneck. That’s why integration is becoming more important.

Then comes the next constraint: infrastructure. Datacenters are no longer just server farms - they are power-constrained, heat-constrained systems. And this is where the conversation expands again - into space. Not as an immediate solution, but as a long-term exploration of where compute could live if Earth-based constraints become binding.

Finally, all of this connects into a broader pattern. Tesla, xAI, SpaceX, and Terafab are best understood not as separate bets - but as different layers of a tightly coupled system.

3. Insights per Deep Tech

3.1 Semiconductors

The most valuable part of a chip is no longer the chip.

For a long time, semiconductor value was easy to locate. It sat inside the chip - better architecture, smaller nodes, higher performance.

But in modern AI systems, that’s no longer where the bottleneck lives. The constraint is increasingly in how chips are combined - especially how compute chips interact with memory.

Take high-bandwidth memory (HBM). It’s not just an add-on - it fundamentally determines how much of a chip’s theoretical performance you can actually use. If memory can’t keep up, the chip spends most of its time idle.

What’s interesting is where this shifts value. Not just to chip designers—but to whoever controls:

• memory integration

• advanced packaging

• system-level layout

In other words, the competitive edge is moving from designing the fastest chip to designing the most efficient system around it.

And that’s a very different game.

3.2 Terafab + Datacenter

The scarcest resource in AI may not be chips - it may be coordinated capacity.

When people talk about scaling AI, they often talk about more GPUs.

But the harder problem isn’t adding more hardware—it’s making everything scale together. You don’t just need chips. You need chips that arrive on time, are packaged correctly, can be deployed into systems, powered reliably, cooled efficiently, and networked together without bottlenecks.

Each step is manageable on its own. What’s hard is synchronizing all of them.

This is where the idea behind something like Terafab becomes interesting - not as a “better factory,” but as a way to reduce coordination friction across the entire pipeline. Because at scale, delays and inefficiencies don’t add—they multiply.

And the companies that win may not be the ones with the best individual components, but the ones that can align the entire system most tightly.

3.3 Spacetech

Compute is becoming a location-sensitive decision for the first time.

Historically, computing had a kind of abstraction. It didn’t really matter where your servers were, as long as they were connected.

But that abstraction is starting to break. As AI infrastructure scales, physical constraints—power grids, cooling environments, land availability - start to matter more. Which means location starts to matter more. Space is one extreme version of this idea.

But even before that, we’re already seeing it on Earth:

• datacenters moving closer to cheap energy

• clusters being built around specific geographies

• infrastructure decisions shaped by physics, not just latency

So the real shift isn’t “we’re moving to space.” It’s that compute is no longer location-agnostic.

And once location matters, infrastructure strategy becomes much more complex - and much more strategic.

3.4 Overall Industry Structure

Control is shifting from layers to interfaces.

For decades, the semiconductor industry was organized in layers. Design → manufacturing → systems → deployment. Each layer created value independently.

But what’s emerging now is something different. The most strategic control points are no longer the layers themselves—but the interfaces between them.

• Between chip and memory

• Between hardware and datacenter

• Between infrastructure and deployment

That’s where performance is won or lost. And that’s where dependency becomes risky.

As I mentioned earlier in this article, this is what makes the current shift feel so unusual - especially from the perspective of someone who has worked inside this industry.

I used to think full integration was unrealistic. And it still might be.

But what we’re starting to see isn’t full integration - it’s something more targeted.

Companies pulling control over specific interfaces where:

• bottlenecks are forming

• dependencies are highest

• differentiation is hardest to outsource

And that’s what makes this moment feel both impressive and slightly unsettling. Because if control moves to those interfaces, then the balance of power in the industry doesn’t just shift slightly. It reshapes entirely.

Mini Section: 🚀 How to become a SpaceX shareholder 💸

▪️ There’s basically no clean way for retail investors to buy SpaceX. It’s still private. Secondary markets exist, but they’re opaque and dominated by institutions

▪️ So the market starts creating proxies. A closed-end fund holding private AI/defense names recently traded at ~1,550% premium to NAV → investors paying 15×+ just for indirect exposure

▪️ In another case, a fund with $19 NAV traded at ~$315 → effectively paying $16 to get $1 of SpaceX exposure

▪️ More workarounds emerging:

• funds tracking private unicorn indices without owning shares

• structured deals mimicking exposure via swaps

• companies like EchoStar rising 300%+, driven more by SpaceX linkage than fundamentals

▪️ This isn’t just hype - it’s a structural signal.
→ the most important AI/infrastructure companies are staying private longer
→ access itself is becoming scarce

▪️ The same pattern we see in compute is now showing up in capital markets
scarcity → bottlenecks → premiums

▪️ Not just who can build the future → but who can own it

4. What’s Real vs What’s Still Speculative

There are real signals here.

AI is pushing against physical limits - chips, power, cooling. Companies are starting to think in full-stack terms. Integration is becoming strategic.

But there’s also a lot that remains uncertain.

Building a $20–25 billion fab is one thing. Building one that competes at 2nm, scaling from 100,000 wafers per month to 1 million, is something else entirely. And orbital datacenters, while conceptually compelling, are still far from proven at scale.

So the architecture of the idea may be real. The execution is still a very open question.

5. What This Means

This isn’t really about Terafab. It’s about how the AI race is being redefined.

We used to think the winners would be those with the best models. Now it’s starting to look like the winners may be those who control the infrastructure beneath them.

Not just chips. But power. Cooling. Manufacturing. Even geography.

The center of gravity is shifting - from software to systems.

6. What Changed My Mind

A year ago, I would have been much more certain about how this industry works. But now I have so many questions.

• Can a new entrant realistically build and operate an advanced-node fab at scale?
  (Given what I’ve seen Elon has done before… I’m more open to it than I would have been three years ago.)

• Will selective vertical integration actually create durable advantage - or will the complexity eventually outweigh the benefits?

• And is space-based compute a real future - or a compelling narrative layered on top of more grounded constraints?

But stepping back, I think the more important realization is this. I used to think the structure of this industry was fixed. That specialization was not just efficient - but inevitable.

Now I’m not so sure. Because what’s changing isn’t just technology. It’s where the bottlenecks are. And when bottlenecks shift, the structure of an industry tends to follow.

So maybe this isn’t really about Terafab or even about Elon Musk. Maybe it’s about a deeper transition: from a world where intelligence was built on modular software stacks… to one where it depends on tightly integrated physical systems.

And if that’s true, then the question isn’t just: Who builds the best models?

It’s: Who controls the system that makes those models possible.

I didn’t grow up thinking CES was some mythical temple of technology.

During my McKinsey projects in deep tech, CES was simply one of many reference points - along with academic papers, startup demos, and customer interviews—when we tried to understand where a technology actually stood. But every January I would watch the announcements and feel that mix of curiosity and distance: frontier ideas appearing on a stage I had never seen in person.

So CES slowly became less of a benchmark and more of a quiet wish:
one day, go and see it myself.

When CES 2026 announced the theme “Innovators Show Up,” I finally decided to stop watching from a laptop screen. I bought the ticket, sent that slightly uncomfortable email to my professor, and flew to Las Vegas expecting a parade of futuristic concepts.

What I found was different.
Less spectacle than I imagined - and much more practicality.

🇺🇸 Silicon Valley - Where AI Starts Doing Chores

Dexterity: Function Before Charm

The first demo that held me was Dexterity’s warehouse system - two large arms on a mobile base stacking irregular boxes into a truck. No face, no attempt to look human, just mechanical confidence.

The operator explained:

“Two workers for 30 minutes → one robot for 10.”

As someone raised on beautifully designed Korean electronics, the deliberate ugliness felt strange. But the logic was hard to argue with: this robot existed to earn its salary, not to win design awards.

Matternet: Drones That Already Have Customers

Matternet’s booth added another layer of realism. They weren’t talking about hypothetical pilots; they were describing active routes in California - medical samples between hospitals, grocery deliveries in suburban neighborhoods, contracts with municipalities.

The business model sounded more like a logistics company than an aviation startup: per-delivery pricing, service-level agreements, fleet maintenance schedules. It was refreshing to hear a robot company speak the language of operations rather than imagination.

Psyonic: When Price Becomes the Innovation

At Psyonic I was reminded that commercialization can be as simple - and as difficult -as lowering cost. Their Ability Hand used to cost tens of thousands of dollars; through new manufacturing and AI-based control it has dropped to roughly $10–20k, opening insurance coverage and everyday adoption.

Watching someone hold a cup and feel pressure through vibration was a quiet but powerful example of value creation - needs first, technology second.

NVIDIA - The Invisible Spine of Physical AI

Among all these individual machines, NVIDIA felt like the city planner.

Their story wasn’t one robot but the stack that makes robots possible: chips, simulation, reasoning models, and deployment tools. Jensen Huang framed this as a “Physical AI moment,” and on the floor I began to understand why.

The new Rubin platform focused on cheaper, faster real-time inference - crucial when a robot must react instantly rather than wait for the cloud. The Alpamayo model family aimed at reasoning about actions, not only recognizing objects. Robots, they argued, need judgment more than sharper eyes.

What interested me most were tools like Cosmos and OSMO for synthetic data and training loops. Real robot data is expensive; NVIDIA is trying to industrialize learning itself. I saw these systems powering demos from Boston Dynamics, Caterpillar, LG, Franka, and NEURA - an entire ecosystem orbiting one middleware layer.

It struck me that NVIDIA isn’t selling a chip.
They’re selling the operating system for a profession of robots.

📈 From Demonstrations to Business Models

Not every booth had this clarity. Many companies still looked like they were rehearsing for a future without customers.

And that’s fine - CES has always been part dream, part spreadsheet.

What felt new was how many conversations revolved around unit economics rather than hero videos: interventions per hour, utilization, service contracts. Physical AI was beginning to sound like operations, not science fiction.

Why Now?

Three trends aligned:

1. Multimodal models connecting vision and language

2. Hardware costs pushed down by smartphone supply chains

3. Labor gaps in logistics, care, and delivery

Together they turned robots into financially legible workers.

🇰🇷 Korea - More Strategic Than I Expected

LG: Turning the Home Into Robot Infrastructure

LG’s CLOi-D booth was packed, and not just because the robot looked friendly. What stood out was the system thinking behind it.

Most home-robot demos try to squeeze a machine into today’s messy kitchens. LG flipped the premise:
redesign the environment first.

Their demo apartment showed:

• fridges with standardized handle geometry

• cabinets positioned for robotic reach

• appliances exposing structured “task APIs”

The robot was careful and sometimes slow, but the idea was larger: CLOi-D as an orchestrator across appliances LG already dominates. From a strategy lens this felt very Korean; use existing market power, control the environment layer, and let the robot improve gradually on top.

www.youtube.com › watch

I did notice limits: manipulation relied on known object locations and recovery was manual. Yet as a first step toward a “Zero Labor Home,” it felt grounded rather than theatrical.

Hyundai Atlas: Legs as Enterprise Software

I entered the Hyundai/Boston Dynamics area skeptical. Wheels seemed more rational than legs.

Atlas complicated that view. The demo focused on workflows:

• balancing asymmetric loads

• stepping over uneven floors

• two-handed tool alignment

Watching it move, I realized the obvious: factories and ships are built for human proportions. Rebuilding all of that for wheeled robots might cost more than teaching robots to use legs.

Hyundai spoke about Georgia plant pilots and RaaS contracts - language closer to industrial automation than spectacle. Atlas began to feel less like a stunt and more like an enterprise roadmap.

A Korea Pattern I Didn’t Expect

Seeing LG and Hyundai together changed my perspective:

• LG → control the domestic environment

• Hyundai → control the industrial environment

• startups → supply specialized brains and components

Instead of chasing one heroic humanoid, Korea seemed to be assembling an ecosystem play across appliances, cars, batteries, and telecom.

🇰🇷 Korean Startups - The Conversations I Remember

WIRobotics: From Wearables to Humanoids

I knew WIRobotics from their WIM S walking assist and appreciated how they started with real mobility needs. At CES they introduced ALLEX, a humanoid concept. The professor and CEO Yongjae Kim at the booth explained how their hand design emphasized compliance and human-like manipulation, and mentioned collaboration with Realworld Labs in Korea.

The speed from assistive devices to humanoids surprised me. It felt like a natural evolution: help humans walk → learn about bodies → build bodies.

Tommoro Robotics: Brains Before Muscles

I had known Tommoro Robotics mainly from their work on the Robot Foundation Model (RFM) and a few logistics pilots in Korea. Seeing them inside the HUMANOID M.AX Alliance booth at CES reframed them from an interesting research-driven startup to a company chasing very concrete problems.

The demo was refreshingly practical: the humanoid RB-Y1 picked a specific item from mixed objects and transferred it onto a rail system after a feeder robot delivered a box. No acrobatics - just a task that looked like something a warehouse would actually pay for. The team shared that similar workflows are already being tested with a major Korean logistics company and a cosmetics manufacturer.

What I liked was the philosophy: “One Brain, a Thousand Bodies.” Tommoro isn’t trying to sell a humanoid as a marvel, but automation powered by a model that can be reused across different forms. The environment was still structured, but the direction - software-first, task-first - felt realistic.

🇨🇳 China - Forget human-wave tactics; this is robot-wave tactics

I believe the Chinese robotics presence deserves its own piece on pricing and iteration speed, so I’ll cover that separately soon!

What CES Adjusted in My Head

1) Form Factor Humility

Wheels often beat legs. Economics beat ego. The most convincing systems fit the task rather than imitated the human shape.

2) Robots Are Platforms, Not Products

A robot is closer to cloud software than to a toaster: data pipelines, updates, ops teams, service contracts. Hardware is only the beginning.

3) Korea’s Real Opportunity

Korea designs the environments robots will enter - apartments, appliances, cars, factories. That ecosystem may matter more than a single breakthrough model.

On the Flight Back

One demo showed a robot folding a towel. It was slow and dropped the fabric once. Nothing cinematic.

Yet on the flight home I kept thinking about it. AI had stepped out from behind glass and tried to help, imperfectly but seriously.

As a Korean student balancing Silicon Valley lectures and Seoul news feeds, I felt less like a spectator and more like someone watching the first day of a new profession - one that might hire both machines and people like me.

CES 2026 wasn’t about robots becoming human.
It was about AI finally becoming useful.

Google released Gemini 3. Benchmark leaderboards reshuffled. Developers quietly changed their defaults. Enterprise conversations stalled mid-decision. And then - almost on cue - reports surfaced that OpenAI had declared an internal “Code Red.” (update: and released GPT 5.2 few days ago!)

If that phrase rings a bell, it should. In early 2023, Google was the one sounding Code Red after ChatGPT blindsided the company and triggered what was widely seen as an existential crisis.

Two years later, the alert has changed hands.

At first glance, this looks like another episode in the AI model arms race: better reasoning, better coding, better benchmarks. But that framing misses what’s actually happening. What we’re witnessing isn’t a model war—it’s a systems war. And Google’s resurgence explains why OpenAI is suddenly under pressure.

To understand OpenAI’s Code Red, you have to rewind—not just to Gemini 3, but to Google’s lowest point.

2023: when Google lost the narrative

I remember the Bard launch vividly.

At the time, I was working on a McKinsey project that touched imaging sensors and edge AI. ChatGPT had just exploded, and when Google announced Bard, I was genuinely excited. Finally, Google - the company that invented the transformer - was stepping back into the spotlight.

I started testing Bard almost immediately, feeding it questions about CMOS sensor architectures and ISP pipelines. Within minutes, the excitement faded.

At one point, Bard confidently explained that a smartphone image sensor “temporarily stores photons in on-chip DRAM before uploading them to the cloud for neural enhancement.” Another answer suggested that rolling-shutter distortion occurred because “pixel rows fail to synchronize on a distributed ledger.”

I laughed. Then I stopped testing.

This wasn’t just hallucination - it was un-Google-like. For a company that trained generations of ML researchers, Bard felt rushed, fragile, and oddly shallow. The now-infamous demo error that wiped roughly 7% off Google’s market cap only reinforced what many of us felt: Google had the research, but not the product system.

And that distinction mattered.

Google didn’t lack tech - it lacked activated tech

It’s tempting to say Google “had the tech but not the product muscle.” That’s directionally right, but incomplete.

Google didn’t lack intelligence. It lacked activated, unified, product-grade intelligence.

Before ChatGPT, Google had:

• World-class model architectures

• Massive training infrastructure

• Deep multimodal data

• Multiple internal LLMs competing for ownership

What it did not have was:

• A single, externally accountable AI product

• A production-ready inference + safety + UX stack

• Organizational permission to ship something imperfect

• Clear default surfaces for distribution

OpenAI didn’t out-invent Google. It out-assembled it.

ChatGPT wasn’t smarter than Google’s internal models in 2022 - but it existed, shipped, and learned from users. Bard, by contrast, was an integration Google wasn’t yet ready to own end-to-end.

That’s why Bard felt brittle. It wasn’t a failure of research. It was a failure of system readiness.

The reset: five structural bets

Google’s recovery wasn’t a single clever fix. It was a coordinated reset across the company.

(1) First, Google Brain and DeepMind were merged. For nearly a decade, two elite AI orgs operated in parallel - brilliant, but duplicative and competitive. The merger wasn’t about org charts; it was about collapsing internal friction and forcing focus.

(2) Second, 12,000 layoffs. Historically taboo at Google, the cuts created space to reallocate talent aggressively toward AI. Crisis gave permission to move.

(3) Third, founder gravity returned. Sergey Brin re-engaged deeply in Gemini’s development. In large tech companies, founders don’t just advise - they compress indecision. Their presence changes risk tolerance.

(4) Fourth, Gemini became the single AI brand. One name for the model, the consumer assistant, and the enterprise offering. This reduced cognitive load and concentrated demand.

(5) Finally - and most importantly - Google doubled down on full-stack AI:
models, custom silicon (TPUs), data centers, cloud, consumer products, and distribution via Android, Chrome, Search, YouTube, and hardware partnerships.

Gemini wasn’t built to win a leaderboard. It was built to power an ecosystem.

Why Gemini 3 changed the conversation

When Gemini 3 landed, the reaction was different.

On LMArena, it debuted at the top, surpassing OpenAI’s latest models in reasoning and coding. But more telling was the qualitative feedback. Developers described Gemini as “thinking before answering.” Slower, yes - but more reliable on multi-step tasks.

This wasn’t accidental. Gemini 3 leaned into explicit internal planning modes, trading latency for correctness. For enterprise use cases - coding, math, research - that tradeoff matters.

Two structural advantages amplified this leap.

First, multimodal data. Google owns YouTube. No competitor comes close. Gemini’s strength in image and video understanding isn’t magic - it’s structural advantage.

Second, silicon-model co-design. Gemini 3 was trained entirely on TPUs, not NVIDIA GPUs. This allowed Google to optimize architectures around its own hardware constraints, improving performance per watt and inference economics.

That combination - better reasoning, multimodality, and cheaper scaling - shifted perception fast.

When product wins, everything unlocks

It’s easy to over-index on org changes. But none of them would have mattered if Gemini hadn’t crossed a real quality threshold.

The flywheel only restarted when people wanted to use Gemini.

Once that happened, everything unlocked.

TPUs - long dismissed as internal science projects - suddenly looked credible to external customers. Companies like Meta and Anthropic began seriously evaluating TPU-based training and inference. Google Cloud’s AI story tightened: Gemini wasn’t just on GCP; it was of GCP.

Consumer products followed. Gemini’s quality made it safe to embed deeply into Search, Workspace, and Android. That mattered enormously. Distribution only works when the product doesn’t embarrass you.

This is the subtle but critical point: Google’s distribution advantage only becomes an advantage after product quality clears a bar. Until then, scale just amplifies failure - as Bard painfully demonstrated.

Once Gemini crossed that bar, the logic flipped. Distribution accelerated learning. Learning improved the model. Improved models justified more TPU investment. TPU investment lowered marginal costs. Lower costs made bundling feasible. Bundling drove consumer usage. Consumer usage justified hyperscale capex.

Product quality wasn’t just one variable. It was the unlock condition.

From the GPU era to the ASIC era

Underneath all of this is also a deeper structural shift: we’re moving from a GPU-dominated AI era to an ASIC-dominated one.

NVIDIA GPUs were the perfect accelerator for AI’s explosive phase - flexible, powerful, and available. OpenAI’s rise is inseparable from that ecosystem.

But GPUs are general-purpose. As workloads stabilize and inference volume explodes, general-purpose becomes inefficient.

Google understood this early. TPUs were never about beating NVIDIA on peak performance. They were about control: predictable supply, tighter software integration, and better performance per watt at hyperscale.

Gemini 3 training entirely on TPUs is not a footnote - it’s a signal. Google believes its custom silicon is now good enough to anchor frontier models, not just internal workloads.

If that belief holds, the implications are profound.

Cost curves bend. Dependence on external suppliers weakens. Competitive advantage shifts back toward companies that own the full stack.

This mirrors earlier transitions: AWS’s Graviton didn’t kill Intel overnight, but it permanently changed the server CPU market. Hyperscalers don’t need to replace GPUs everywhere - just enough.

For AI, custom silicon is how intelligence becomes cheap, reliable, and ubiquitous.

Why OpenAI is in Code Red

Seen through this lens, OpenAI’s Code Red isn’t about ego or benchmarks. It’s about math.

AI is commoditizing faster than expected. Open-source pressure from China is real. Performance gaps are narrowing.

At the same time, costs are exploding. Even optimistic scenarios suggest OpenAI needs hundreds of billions in additional capital to sustain growth. Planned data-center investments alone stretch toward the trillion-dollar range.

Worse, OpenAI lacks consumer amortization.

ChatGPT is an app users choose. Gemini is increasingly an assistant users inherit - preloaded on Android, embedded in Search and Workspace, and soon hardware like smart glasses (I’m so excited about these glasses and will write a separate article on it!!!!)

This is why Anthropic looks calm. Anthropic chose a B2B-first path with tighter scope and lower free-user load. OpenAI chose scale - and now has to feed it.

Code Red isn’t about Gemini being “smarter.”
It’s about Google activating a system OpenAI can’t replicate quickly.

From model wars to systems wars

The real competition now isn’t GPT vs Gemini vs Claude.

It’s full-stack vs partial-stack AI.

Google controls training hardware, inference economics, consumer distribution, enterprise cloud, and personal context. OpenAI controls a beloved interface and world-class research velocity - but not the full loop.

Once AI becomes infrastructure-heavy, systems win over features. We’ve seen this before in compute, mobile, and cloud.

AI is entering that phase now.

Closing thought

Two years ago, Google looked slow and complacent. Today, it looks methodical - and dangerous.

OpenAI’s Code Red is understandable. But it’s also revealing. It signals that the game has shifted from sprinting ahead on models to building systems that compound.

Google didn’t win by inventing intelligence first.
It’s winning by turning intelligence into infrastructure.

And that’s the real lesson of this moment.

In the last piece, we stayed inside the robot’s head. We looked at how teleoperation, massive simulation, and Vision–Language–Action (VLA) models are slowly giving robots something they never had: an internet of actions to learn from, instead of the web-scale internet of text that powered large language models. That was the “how robots actually learn” story - the data flywheel and the emerging robot brain.

This article picks up right where that one left off. Once you do have that brain and that training pipeline, a new question appears:

How should the world use it?

For decades, programming a robot has felt less like asking for help and more like negotiating with a very stubborn machine.

To get a simple task done - say, clearing a table - you don’t just say “clear the table.” You specify which arm to use, how fast to move each joint, where to grip each object, how much force is safe before something slips or breaks, how to avoid collisions with chairs and people, and what to do when the world doesn’t look exactly like the CAD model. Every variation in layout, lighting, or furniture creates a new pile of edge cases.

This is why most robots have lived in tightly controlled bubbles: factory lines with fixed fixtures, fenced-off workcells, carefully scripted motions. When everything is bolted down and nothing changes, you can afford to handcraft the “how.” As soon as you step into an open-ended café, warehouse, or hospital, that approach collapses under its own complexity.

Software went through a similar phase. Early on, apps had to know far too much about the systems around them: different operating systems, bespoke protocols, one-off integrations. The web didn’t take off because we made every interaction more detailed; it took off because we hid detail behind clean interfaces. You stopped worrying about exactly how someone else’s server worked and started trusting that a simple call would get you what you asked for.

The same kind of abstraction is starting to appear in robotics.

In #7, we stayed with the training stack - teleop, sim, VLA, data flywheel. In #8, we zoom out one layer and treat that stack as a service. Instead of thinking about robots one by one, we think about capabilities - “clear this aisle,” “scan these shelves,” “reset this rack” - that can be invoked without micromanaging every joint.

That’s the idea behind a Physical API.

1. What we mean by a Physical API

In software, an API hides complexity. You express what you want done; the service decides how to do it.

A Physical API is the analogous idea for physical work:

• You express a goal in a structured way:
  “clear this table,” “restock this aisle from this inventory,” “assemble this rack to this spec.”

• You attach constraints and preferences:
  timing (“finish before opening”), safety rules (“never cross this line”), priority (“this row first”), quality thresholds (“no visible crumbs / defect rate < X%”).

• The system decides which robots to use, which skills to invoke, and which trajectories to execute. It also monitors the task, surfaces metrics (“success rate,” “average cycle time”), and recovers from small errors when it can.

You’re no longer thinking, “How do I program this specific arm?” You’re thinking, “How do I call the ‘clear table’ capability in this space?”

Under the hood, the Physical API layer has to:

• translate goals into skills (pre-trained behaviors) that exist in the robot brain,

• ground those skills in a concrete environment using maps and perception,

• schedule and route work across multiple robots,

• and keep operators in the loop when something ambiguous happens.

VLA models and world models provide much of the intelligence; the robots are the actuators behind that interface. The operator interacts with the API surface, not the joint-space details.

2. Smarter AI → simpler, cheaper hardware

Historically, industrial robots had to be extremely precise and predictable. The “intelligence” lived in:

• humans who designed fixtures and jigs,

• engineers who hand-tuned trajectories and safety envelopes,

• and offline planning tools.

The robot itself was a very accurate player piano.

As perception and control models improve, you can tolerate less perfect, more affordable hardware:

• Slight mechanical slack can be compensated by feedback control.

• Variations in part placement can be handled by vision rather than hard stops.

• Small misalignments can be corrected on the fly instead of causing a fault.

We’ve already seen a broad cost movement:

• from six-figure research platforms,

• to five-figure industrial arms in many factories,

• to simpler systems and components in the mid- to low-five figures (with some elements dipping into the high four figures), depending on spec and volume.

Exact numbers vary by vendor and configuration, but the direction is clear: if most of your “intelligence” lives in shared models and software, the marginal cost of adding another robot endpoint to your Physical API can keep dropping over time.

That doesn’t mean hardware becomes trivial. Reliability, safety, maintainability, and supply chain still matter. But it means you can start to think about robots less as bespoke capex projects and more as programmable terminals that sit behind the same logical interface.

3. Forward looking into the “Skill Economy”

Once you have:

• a reasonably standardized brain (a VLA-style model trained on your “internet of actions”),

• a growing base of shared experience across tasks and environments,

• and fleets of compatible robot bodies,

it’s natural to imagine a skill economy forming.

A “skill” here is more than a macro. It could encapsulate:

• how to make a specific drink across a family of cafés (different machines, same end result),

• how to execute an inspection routine in different data centers (different rack layouts, same checklist),

• how to prep a hospital room according to protocol (different room shapes, same infection-control requirements),

• or how to assemble a particular product in a light manufacturing cell.

Each skill embeds:

• tacit expert judgment: what “clean enough,” “tight enough,” or “properly aligned” actually looks like in that domain,

• safety checks and fallbacks: when to stop and ask a human, when to log an anomaly, when a hazard threshold is crossed,

• language hooks: natural-language controls like “do it more gently,” “skip dusting the top shelves at night,” “prioritize this side first.”

In practice, skills might be:

• authored initially via teleop + sim + offline tuning,

• packaged as reusable modules inside the robot software stack,

• parameterized for different customers or facilities (“this SKU list,” “these aisle lengths”).

We’re not at a mature skill marketplace yet. But if the ecosystem keeps maturing, you can imagine:

• manufacturers shipping skills tuned to their own equipment,

• service companies encoding their SOPs as skills that work across many sites,

• third-party developers specializing in niche skills (e.g., “museum artifact handling”),

• and operators selecting and configuring skills the way they adopt SaaS integrations today.

At that point, a Physical API starts to feel like a real platform, not just a metaphor.

4. From buying robots to buying “tasks as a service” (my imagination)

Today, bringing robots into an operation often looks like:

• buying or leasing expensive hardware,

• hiring or contracting a robotics team,

• doing months of integration and safety validation,

• then living with whatever capabilities you managed to hard-wire in.

It feels a lot like buying and running your own servers in the pre-cloud era.

A mature Physical API opens the door to different business models. This part is speculative, but directionally plausible:

• Instead of buying robots outright, customers subscribe to a set of physical capabilities:
  “nightly aisle scans,” “per-shift pallet moves,” “rooms cleaned per day.”

• They pay based on outcomes or usage - the work done - not just the list price of hardware.

• Providers handle hardware selection, maintenance, and skill updates behind the scenes, much like cloud providers upgrade their infrastructure.

Under the hood, there are still technicians, spare parts, replacement mops, and blocked drains. But from the customer’s perspective, they’re buying completed tasks, not machines.

Early hints of this model already exist in some cleaning, delivery, and warehouse-automation contracts: customers are sold uptime and throughput, not just units of hardware. The Physical API concept pushes that logic further, into a world where a single abstraction layer coordinates many types of robots and skills across many sites.

5. CosmicBrain AI: one example of where this is going

CosmicBrain AI is one of several startups that are explicitly trying to build towards this Physical API world.

Publicly, they describe themselves as a “cursor for robots”: an all-in-one, no-code/low-code environment for creating, testing, and deploying robot skills.

A few things stand out from their materials and interviews:

• Skill-creation IDE. They offer a visual programming environment and drag-and-drop interface so non-expert users can define robot behaviors and task sequences without deep robotics coding. Think of it as a design studio for skills rather than for single robots.

• Simulation-first workflow. Their platform integrates simulation tools so that skills can be tested and validated before touching real hardware, leveraging physics engines and world models to shrink the gap between sim and real.

• Skills as APIs, not data sales. In one IBM interview, the founder sketches the ambition as an “app store for robots,” but with a twist: instead of selling data, they aim to sell skill sets exposed as APIs, so that customers keep control of their own data while tapping into shared capabilities.

• Shared foundation, reused intelligence. In public posts they emphasize that robots shouldn’t have to relearn everything from scratch for each deployment; a shared foundation and reusable skills should carry over across customers and fleets.

CosmicBrain is still early, and the market is young. But they’re a useful concrete example: not just building another robot, but trying to build the tools and marketplace layer that makes a Physical API usable - and commercial.

They won’t be the only ones. If the Physical API thesis is right, we should expect:

• IDEs and toolchains for skill authoring,

• marketplaces for distributing and monetizing those skills,

• and orchestration platforms that let enterprises plug multiple hardware vendors into a single abstraction layer.

CosmicBrain is one of the first teams whose branding and product explicitly point in that direction.

6. Macro: robots and AI’s capex wall

Zooming out, this has implications beyond any one startup.

Most forward-looking AI roadmaps over the next decade involve massive capital expenditure:

• more and larger data centers,

• chip fabs and advanced packaging facilities,

• power generation and transmission,

• solar and wind build-out,

• logistics networks to keep all of this supplied.

Even if you can finance all that infrastructure, you still hit a constraint: human labor to build, deploy, inspect, and maintain it.

If robots with useful autonomy reach reliable operation in key niches - warehouses, plants, solar fields, data centers - within the next decade, they could become a significant enabler for scaling AI and clean-energy infrastructure. Not the only lever, but a meaningful one.

In that world:

• The internet of actions is the data and training fabric that makes robots competent.

• The Physical API is how we expose that competence to the rest of the economy.

• Skill platforms and marketplaces are how we distribute it.

The abstraction layer we’ve been talking about stops being a thought experiment and becomes the way you program the physical side of the AI boom.

7. Conclusion: from learning to leverage

If you read #7 and #8 together, there’s a simple story hiding under the jargon.

Article #7 was about learning:
how robots get their experience when there is no web to scrape; how teleop, simulation, and VLA models slowly add up to an internet of actions; how a data flywheel can, if we’re lucky and careful, push robots toward more robust autonomy.

Article #8 is about leverage:
once you have that brain and that experience, how do you turn it into something the world can actually use? How do you hide complexity behind a clean Physical API, package expertise into skills, and shift from buying metal to buying completed work?

The physical Turing test - “clean the house and make dinner, and I can’t tell who did it” - is still aspirational. The “single-digit years” timeline for broad autonomy is very much a bet, not a guarantee. But the trajectory is clearer than it was a few years ago:

• Data, simulation, and VLAs are giving robots a learning substrate that looks more and more like an internet of actions.

• Platforms like CosmicBrain AI hint at how that substrate might be turned into programmable skills and services.

• And the Physical API framing gives PMs, founders, and investors a way to reason about robots not just as machines, but as infrastructure - a new layer in the tech stack where value, and power, will concentrate.

For builders and investors, the most interesting question is no longer simply:

“Will we have autonomous robots?”

A sharper version is:

“As physical work becomes increasingly programmable, who will own the data, the skills, and the distribution - and what will you build on top of that?”

That is the shared through-line of #7 and #8 - and it’s the real strategic stake in the emerging internet of actions and the Physical APIs that will sit on top of it.

• Large language models grew up with a web-scale internet of text and images. Robots don’t have an equivalent “internet of actions” for physical behavior. Existing robot datasets are tiny and narrow in comparison.

• For many labs pushing general-purpose robots, a big chunk of real-world experience still comes from teleoperation. It’s powerful but slow, expensive, and fundamentally capped by human time.

• Simulation is becoming the “clean energy” for robot learning: thousands of parallel physics worlds plus generative tools that spin up endless virtual kitchens, warehouses, and offices.

• If teleop, simulation, and VLA models do click into a self-reinforcing data flywheel, robots can climb a ladder from scripted demos → narrow autonomy → something closer to a “physical Turing test.”

LLMs had the internet. Robots don’t.

Large language models had an unfair advantage: they were born into a world where billions of people had already poured text, images, code, and conversations onto the web for decades.

Turning that mess into training data was non-trivial, but the raw material was there. The internet acted as a fossil record of human cognition.

Robots don’t have the same luck.

There’s no public, web-scale dataset of “how to unload a delivery truck,” “how to reset a breaker in a data center,” or “how to clean up after a hackathon and cook a candlelit dinner.” We have videos, but not the thing a robot actually needs: continuous joint angles, forces, contact events, and fine-grained corrections.

That’s why you hear people in embodied AI talk in terms of a “physical Turing test”:

Walk into a shared house on Sunday night after a chaotic weekend. Someone has cleaned the place and made dinner. You shouldn’t be able to tell if it was a human or a robot.

— Jim Fan, Director & Distinguished Research Scientist at NVIDIA

We are nowhere near that benchmark today. Current humanoids still fumble cereal and cables. But the interesting story is not about one viral demo; it’s about how the field is slowly constructing an “internet of actions” and how, if that succeeds, it naturally leads toward something like a Physical API for the real world.

Where robots are actually stuck: data, not (just) hardware

In top-tier labs and companies working on general-purpose robots, you rarely see a shortage of hardware:

• robot arms, mobile bases, prototype humanoids;

• multiple cameras and depth sensors;

• racks of GPUs in the back.

One major bottleneck is experience - especially diverse, high-quality experience.

1. Teleoperation as “human fuel”

For many cutting-edge embodied AI projects, a large share of ground-truth experience still comes from teleoperation:

• A human wears a VR headset or uses special controllers.

• They see from the robot’s point of view.

• They manually drive the robot’s joints through tasks: take bread out of a toaster, pick up a cup, open a cabinet, pour honey without spilling.

Every one of these sessions produces a rich trajectory: joint angles, velocities, contacts, successes, failures. That’s precious supervision.

But teleop has hard ceilings:

• It’s tiring and attention-heavy.

• It doesn’t scale linearly with money the way scraping websites does.

• In theory, you could teleop one robot 24 hours a day; in practice, you quickly run into human limits.

And every new environment - a different kitchen, warehouse, or hospital - often needs fresh data, because the small details (layout, lighting, fixtures) change everything.

Relative to the web for LLMs, robot learning is still running on very expensive calories.

2. Why “more demos” isn’t enough

You could imagine just doing more teleop, but that doesn’t fully solve the problem.

A genuinely useful robot doesn’t just need:

• the “average” demo under ideal conditions.

It also needs:

• robustness to odd cases (slippery plates, torn packaging, weird clutter),

• the ability to work in slightly different layouts,

• graceful recovery from small mistakes,

• and efficiency and safety across all of that.

That’s not just more of the same data; it’s different kinds of data:

• more diversity in environments,

• more strange edge cases,

• and more repeated practice under systematic variation.

So if there’s no internet of actions to scrape, and human teleop doesn’t scale without limit, where does the rest of the experience come from?

Simulation as clean energy for robot experience

One major answer is simulation.

Early robot simulators looked like basic game engines with physics turned on. Modern ones look more like experience factories: thousands of parallel environments on GPUs, each with randomized mass, friction, lighting, and clutter, all feeding data into policies that never get tired.

The simulation ladder: Sim 1.0 → 1.5 → 2.0

You can think of the field as climbing a simple ladder along one axis: speed vs. diversity.

• Sim 1.0 – Digital Twin: very fast, high-fidelity physics copies of a specific robot and workspace, but environments are mostly hand-built and limited in variety.

• Sim 1.5 – Digital Cousin: still physics-based, but scenes and motions are generated and randomized by models, so you trade some speed for much richer diversity.

• Sim 2.0 – Digital Nomad: robots train inside neural “world models” or video models that can imagine countless futures, slower today but with virtually unlimited diversity as compute and data scale.

Most industrial and academic systems today live somewhere between 1.0 and 1.5. The interesting question for humanoids and Physical AI is how quickly we can move more of the learning signal into 2.0.

From fossil fuel ⛽️ to clean energy 🌿

A helpful analogy:

• Teleoperation is fossil fuel: potent but human-limited. You can get high-quality data, but only as fast as humans can drive robots.

• Simulation is more like clean base-load power: once you’ve built the plant (the physics engine + world model), you can run it continuously.

With Sim 1.0–1.5, you can already run years of practice in hours of GPU time and expose robots to failures you’d never tolerate in a live facility. With Sim 2.0, the “plant” becomes a neural world that can keep inventing new, plausible situations.

The long-term aim is to flip the ratio:

• Use simulation – especially Sim 2.0–style neural worlds – for the bulk of pre-training and diversity.

• Use real-world data and teleop to anchor, correct, and fine-tune.

That’s how the field is trying to manufacture an “internet of actions” where none existed: a shared, ever-growing reservoir of experience that future robots can draw from before they ever touch the real world.

1. Sim 1.0–1.5: fast but capped

In Sim 1.0, the pattern is:

• Build a digital twin of your robot and workspace.

• Use domain randomization – deliberately randomize gravity, friction, object weights, textures, camera noise.

• Train policies on huge amounts of synthetic experience.

This has already given us:

• Quadrupeds that learn to walk, jump, and recover from pushes in sim, then transfer to real hardware.

• Manipulators that learn to spin objects, re-grasp tools, or open doors after seeing many stochastic variants.

Sim 1.5 pushes diversity further by letting generative tools help build the world:

• Language describes tasks and layouts: “a cluttered sink with dishes, a drying rack, a sponge, and a bottle of soap.”

• 3D generators populate scenes with varied objects and materials.

• Image models synthesize realistic textures and lighting.

The robot model and core physics stay grounded; almost everything else is procedurally created. Instead of hand-authoring a few canonical scenes, you can train across thousands of plausible kitchens, offices, and warehouses that nobody manually modeled.

But even with 1.5, you are still fundamentally authoring environments and pushing data into the simulator. Diversity expands, but not infinitely.

Source: NVIDIA

2. Sim 2.0: robots living inside neural worlds

Sim 2.0 flips the perspective: instead of manually building more and more complex 3D worlds, you train a neural world model that is the world.

In practice, Sim 2.0 looks like:

• A large video or 3D generative model that has digested millions of real-world clips: kitchens, offices, warehouses, streets.

• The model can roll out many possible futures from a given state: the same object picked up with a different grip, in slightly different lighting, with a different obstacle in the way.

• A robot policy interacts inside this “dreamspace,” trying actions, receiving feedback, and learning how the world tends to respond.

Instead of saying “here is a perfectly modeled warehouse,” we say “here is a world model that knows 10,000 ways warehouses can vary” – and we let the robot be a digital nomad, wandering through that distribution.

For humanoids and Physical AI companies, this matters because:

• Diversity beats hand-crafting. Edge cases become the norm: spilled coffee, odd furniture, awkward human poses, half-open doors. These are exactly the situations where brittle policies fail.

• Long-horizon skills become learnable. World models can simulate not just single grasps but entire routines – clean a table, fetch an item, reset a room – and provide gradients over the whole sequence.

• Scaling becomes a compute problem, not a labeling problem. Once the world model is trained, generating more experience is “just” more compute, not more teleoperation hours or manual scene design.

This is the jump from “we simulate a specific factory bay very well” to “we simulate enough of the world’s distribution that a robot can be dropped into a new building and still cope.”

3. Why this is a big deal for PMs and VCs?

If you’re evaluating humanoid or Physical AI startups, Sim 2.0 is not just a technical curiosity; it shapes who can actually compound:

• Compute curve: Who has a credible path to scaling world-model training and simulation rollouts as GPUs get cheaper or more specialized? Are they treating simulation like a first-class compute workload, or an afterthought?

• Data flywheel: Who has access to rich, real-world video and robot logs to train world models – and a plan to keep that data flowing (deployments, teleop, partnerships)?

• Standards and tooling: Are they building on an ecosystem that can become a de facto standard (APIs, scene formats, evaluation benchmarks), or are they locked into a bespoke stack that won’t attract partners?

• Platform posture: Are they positioning their simulator / world model as an internal advantage only, or as a platform others can train on (e.g., developers, integrators, OEMs)? The latter tends to generate defensibility and network effects.

In other words: Sim 2.0 is where “simulation as a feature” turns into simulation as infrastructure – the equivalent of a cloud for robot experience.

Vision–Language–Action models: the new robot brain

Data alone doesn’t make a robot useful. You also need a brain that can:

• understand scenes,

• interpret instructions,

• and produce actions over time.

A leading candidate for that role is the Vision–Language–Action (VLA) model.

1. What a VLA model actually is

In simple terms, a VLA model ingests:

• what the robot sees (pixels or other sensor inputs),

• what the human wants (a text instruction, spoken command, or higher-level goal),

• sometimes a history of states/actions,

and outputs:

• low-level motor commands, or

• higher-level action primitives that other controllers can execute.

It’s a generalist controller: see + read → decide → move.

2. Planner + controller: splitting the work

A pattern you see in several modern systems:

• A vision–language backbone (often adapted from a general-purpose model) handles perception and reasoning. It parses the scene, interprets the goal, and plans in terms of sub-tasks: “pick up the sponge, then the plate, then rinse.”

  

• A high-frequency action expert - typically a smaller network trained heavily in sim and real data - handles the millisecond-scale control needed to execute those sub-tasks.

Architecturally, many of these systems resemble a kind of mixture-of-experts: different components specialize in planning, grasping, locomotion, etc., while sharing a common representation of the world.

3. Language as supervision, not just interface

As these models get better, language starts to play two roles:

• Interface: humans can say “wipe down this counter” instead of specifying trajectories.

• Supervision: humans can talk while they demonstrate, and those utterances become labels.

Imagine teleoperating a robot and narrating:

• “Now grab the handle.”

• “The mug is slipping; re-grasp from the side.”

• “Push more slowly until you feel the click.”

Those phrases can be aligned with the motion data. Over time, the system can internalize:

• what “re-grasp” means in different contexts,

• how “gently” changes contact forces,

• and when to apply those corrections on its own.

This is still an active research frontier, but the direction is clear: humans become both puppet masters and teachers of the internet of actions.

From teleop to data flywheel to physical Turing test

Putting these pieces together, you get a loop:

1. Teleoperation and scripted policies provide the seed dataset and safe starting behaviors.

2. Simulation amplifies that seed into millions of varied experiences, including edge cases.

3. VLA models distill all that into reusable, language-guided skills.

4. On-the-job learning and human feedback refine those skills further in real deployments.

That loop is what we’re calling the data flywheel.

Every new deployment - whether in a warehouse, a backroom, or a lab - doesn’t just use existing skills. It also:

• exposes the system to new layouts and tools,

• surfaces novel failure modes,

• and produces more data that can be folded back into training.

In that picture, autonomy doesn’t flip from “off” to “on” overnight. It expands gradually:

• Phase 1: robots reliably execute a few tight loops (fold boxes, shuttle items, run a simple inspection).

• Phase 2: they handle small variations and recover from minor errors.

• Phase 3: they string together multi-step routines (“close down this station,” “reset this rack,” “prep this room”).

• Phase 4 (farther out): they approach something like the physical Turing test at least in constrained domains.

Some prominent researchers have publicly speculated that, if current trends in data, sim, and models continue, we could see robots performing genuinely useful, multi-step loops in everyday environments within single-digit years. Others are more cautious and emphasize remaining gaps in safety, reliability, and sim-to-real transfer.

Either way, if the flywheel starts spinning, robots begin to look less like fragile prototypes and more like dependable infrastructure for specific slices of the physical world.

That’s where the Physical API framing starts to make sense.

Conclusion

The surface story in robotics is still dominated by hardware - new humanoids, new demos, new grippers. But underneath, the real shift is epistemic: we are, piece by piece, building something like an internet of actions from teleoperation logs, massive simulation, and on-the-job learning, then distilling it into Vision–Language–Action brains that can understand both pixels and instructions.

Whether the “physical Turing test” arrives in five years or fifteen, this is the path it will walk along. If you care about where autonomy is really coming from, it’s less about any single robot video and more about this growing, shared pool of experience that future robots will all be swimming in.

It sparked a bigger question: where is Adobe taking the creative stack next? After MAX 2025, the answer looks less like “more features” and more like a creative OS.

How Adobe is turning model choice, trust, and agentic UX into a moat

Adobe’s opening at MAX 2025 was less “a new toy” and more “a new operating model.” The company is steadily reframing Firefly + Creative Cloud as a single workspace where you pick the best AI model for each task, keep provenance intact, and move from intent to finished asset without hopping tools. Practically, that means Firefly now hosts Adobe’s own models together with partner models - Google (Gemini, Imagen, Veo), OpenAI (GPT-Image), Runway, Luma, Topaz, ElevenLabs, and more - “in one place, at one price.” For creators and teams, this isn’t a slogan; it’s a different unit of work: specify outcome → route to the right model → keep control and credits → ship.

What actually changed

Three shifts stood out.

1) Model routing becomes productized.
Inside Firefly and core apps like Photoshop/Express, you can now choose among multiple frontier and specialty models at execution time. Adobe isn’t forcing a single stack; it’s brokering the best model per subtask (e.g., different models for Generative Fill vs. upscaling), and it’s already wiring partner options like Topaz Bloom/Gigapixel into Photoshop’s pipelines for Generative Upscale. This is an aggregator posture with opinionated rails.

2) Personalization moves to the model layer.
Firefly Custom Models (private beta) let you drag-and-drop your own reference images and train a private, on-brand style model. The promise is consistency at scale - campaign families, character systems, house looks - owned by the creator or brand and not accessible to others without authorization. That shifts lock-in from file formats to institutionalized style, which is much harder to rip-and-replace.

3) Agentic UX is arriving in the tools you already use.
Adobe previewed Project Moonlight, a conversational, context-aware “creative director” that pulls from Creative Cloud libraries and even linked social accounts to propose ideas, compose first passes, and keep outputs on-brand. In parallel, AI Assistants land in Photoshop and Express to take on repetitive edits and surface step-by-step help, without removing tactile control. This isn’t a separate chatbot; it’s the UI layer inside the app.

To be honest, the AI Assistant is really appealing to someone like me, who often becomes a designer for the team, but was never trained professionally so don’t know all the shortcuts / cheat keys to design work. Like, ‘rename all my layers’ may seem like a tiny job, but that would save a ton of time in my work with my current robotics club.

The feature rundown

• Firefly Image Model 5 (public beta): native 4-megapixel generation, stronger photorealism (lighting, textures), and Prompt-to-Edit so you can describe edits in plain language; layered image editing is coming (in development). This narrows the “mock → comp → polish” loop.

• Audio joins the stack: Generate Soundtrack (timed to picture) and Generate Speech (multilingual voices via Firefly + ElevenLabs, with controllable emotion/emphasis). It’s a credible “no-copyright-drama” answer for commercial video.

  

• YouTube distribution built-in: a Create for YouTube Shorts space inside Premiere mobile - templates, effects, and one-tap publish to Shorts - puts Adobe tools at the moment of posting, not just production. Expect this to expand their creator funnel.

  

• Partner models, not just partners: Beyond Google/OpenAI/Runway/Luma, Adobe is actively wiring Topaz into Photoshop flows (e.g., Generative Upscale) and ElevenLabs voices into Firefly. This is a concrete interoperability stance, not a press-release logo wall.

  

• Provenance as policy: Content Credentials (the CAI/C2PA “nutrition label”) remain central; with broader industry adoption, this is increasingly an enterprise procurement unlock where provenance is non-negotiable.

Strategy: why this is durable

Choice with trust. Adobe’s bet is that creative work is heterogeneous: one “best” model rarely wins across ideation, edit, upscale, and voice. By routing intent to the right model inside a trusted environment (clear data contracts, provenance badges), Adobe converts fragmentation into a competitive edge. It also blunts single-model platform risk while capturing the UI/workflow layer where value accrues.

Style as lock-in. When a brand’s look lives in a Custom Model that assistants can invoke across Photoshop/Express/Premiere, switching isn’t just moving files; it’s retraining institutional taste. That raises retention and seat expansion in a way point tools will struggle to match.

Agentic UX → lower CAC, higher LTV. Assistants shorten time-to-first-value for newcomers and offload drudge work for pros. If Moonlight meaningfully stitches context across apps and channels, expect higher DAU and project throughput - a subscription flywheel, not a one-off feature bump.

Distribution as a growth loop. The YouTube Shorts lane plants Adobe where creators actually publish. Templates + one-tap posting are subtle but powerful network effects - when the path to audience is paved, upstream creation tends to follow.

What to watch

• Assistant usage inside Photoshop/Express: multi-turn sessions per user/week, not just activation.

• Custom Model adoption: number of teams shipping on a house style model; assistant calls that reference those models.

• Partner-model mix: how often Photoshop/Firefly jobs route to non-Adobe models (e.g., Topaz, Google) vs. Firefly; tells you whether Adobe is truly neutral or nudging.

• Provenance penetration: % of exported assets with Content Credentials across agencies/brands - now feasible given CAI/C2PA ecosystem traction.

Founder/PM takeaways

• Build for intent → routing → control, not a monolithic model.

• Treat style as a first-class artifact (versioned, sharable, testable).

• Meet users at distribution; creation and posting shouldn’t be separate journeys.

• Instrument trust (provenance, permissions, training data posture) as product, not legal copy.

Bottom line: Adobe isn’t winning because it has “more AI.” It’s winning because it’s turning choice, trust, and agentic workflows into a cohesive creative OS. If they keep the rails tight and the ecosystem truly open, that’s a hard position to disrupt.

Let me wrap up this article with Adobe’s official video debriefing what happened at AdobeMAX.

• The internet era was built on ranking (search + recs). The next era is acting: assistants and robots that plan and do.

• Assistants moved from seq-to-seq → chain-of-thought → post-training → agentic workflows, with personalization as the reward model.

• Robots get “general” through a recipe: broad pre-train, curated post-train, protect language, and plan environment diversity.

• Early wins in deployment look like skills portfolios plus human oversight, with reliability and intervention rates driving ROI.

• My takeaway: Assistants plan; robots prove. Reliability, not model size, separates demos from products.

Why this afternoon mattered

(I’ve just been so excited to find this event existed!! Huge thanks to my Taiwanese classmates :))

The internet’s big money machines - search and recommender systems - sorted choices. Now we’re moving to systems that take actions: assistants that coordinate tools and robots that work in messy spaces. UTHF (U.S. Taiwan High-Tech Forum) and NATEA (North America Taiwanese Engineering Association) prepared this event to make this shift feel real.

Event primer

Where & who: Computer History Museum, Mountain View. Keynotes by:

• Dr. Ed H. Chi (VP of Research @ ) on “The Future of Personalized Universal Assistant”

• Professor Chelsea Finn’s (Co-Founder @ , Assistant Professor @ Stanford university) talk on “Bringing AI to the Physical World”

• Dr. Ashish Kapoo (Founder @ and Dr Chi Chiu (Founder @CosmicbrainAI) panel talk on “Scaling from Simulation to Reality”.

Why now: The theme was simple: turn breakthroughs in agentic AI and embodied models into systems that keep promises outside the lab.

TL;DR: What shifted in my head

1. Ranking → Generation → Agency. Assistants aren’t just answering; they’re planning multi-step workflows. Personalization is the reward signal.

2. “Generalist” is a recipe, not a parameter count. Pre-train wide, post-train right, protect language, and budget environment entropy like a test plan.

3. Hierarchies help. A high-level planner translates open-ended prompts into simple steps for a low-level controller; synthetic relabeling scales supervision.

4. Deployment ≠ demo. Early rollouts will look like skills portfolios with human oversight, CI/CD-style ops, and strict filters on synthetic data.

5. Reliability is the moat. Imitation often caps around ~80%; RL-on-top + memory is the real path to 95%+.

“The Future of Personalized Universal Assistant”

Dr Chi took his time to guide us through the journey of how we’ve moved from sequence-to-sequence to chain-of-thought to post-training. That stack pushed us past ranking into token-by-token generation that can chain tools. He showed “Project Astra”, a prototype assistant that reads a manual, finds a YouTube tutorial, pulls info from your email, and calls a store - System-1 patterning plus System-2 planning in one loop. The ingredients he emphasized: multi-step reasoning, agent workflows, synthetic data, and personalization.

Why it matters

Ranking taught us what “good” looks like; agency adds planning and action. The assistants that win will look like Planner (System-2) ↔ Skills (System-1) with a user-grounded reward. Personalization isn’t garnish; it’s the objective. A metric I want teams to publish: goal completions per minute of user attention saved.

Why this matters to Physical AI

A personalized universal assistant is the control plane for the physical world. It keeps a live view of your context (vision, audio, text), remembers preferences and state, and hands structured commands to actuators - robots, appliances, services - through tool APIs. Personalization is the reward model that keeps actions aligned once real things move. In short: assistants plan; robots prove.

How my view evolved

I think many people around me, including myself till a year ago, tend to treat personalization as nice-to-have UX. We should now see it as the optimization target and the clean bridge from software agents to reliable physical action.

“Bringing AI to the Physical World”

Professor Chelsea Finn opened by reframing “generalist robots” as a recipe, not a parameter race - then walked the room through that recipe using concrete cases from her company, Physical Intelligence, with clear visuals: a PaliGemma-based VLM driving a separate Action Expert, demos spanning single-arm, bimanual, and mobile manipulation, and long-horizon laundry tasks that transferred to unseen homes.

Her core insights were pragmatic: generalization is engineered (budget environment entropy like test coverage), real-world data is costly, so make each minute count (reuse legacy static datasets, synthesize supervision via LLM relabeling, add experience-retrieval memory, and use targeted RL when imitation plateaus around ~80%). The headline takeaway: the near-term “generalist” is multi-object and multi-layout inside bounded workflows - ship with a disciplined recipe now, then widen the envelope on a schedule.

What Physical Intelligence has built

• Two-part model. A ~3B vision-language backbone conditions a ~300M Action Expert that emits ~50-step control chunks (diffusion/flow-matching). Inputs: 1–3 images + language; outputs: joint commands for single-arm, bimanual, and mobile platforms.

• Protect language. Naive end-to-end fine-tuning caused the policy to ignore instructions. Their fix: tokenized actions and a stop-gradient barrier from the control head, so the VLM keeps its instruction-following “brain.”

• Recipe > soup. Broad pre-train (robot + web) + curated post-train on gold teleop demos beat “train on all data,” especially on long-horizon tasks.

• Generalization you can engineer. More and more diverse locations → better transfer to unseen homes. Prior static-arm data still helps mobile tasks.

• Open-ended prompting. A high-level planner turns “make a vegan sandwich (no pickles)” into a simple command sequence; LLM relabeling creates instruction–response pairs without expensive human-robot chats for every variant.

Where it still struggles

• Perception & occlusion (esp. thin/transparent objects) → needs better views, sometimes different sensors.

• Deformable manipulation (laundry, bags) → benefits from higher-quality teleop and focused data.

• Long-horizon drift (losing the thread mid-task) → needs experience-retrieval memory.

• Premature “good-enough” stops → responds to RL fine-tuning on those failure clusters.

The cost reality, and how they reduce it

Real-world data is expensive and slow to collect: getting into homes/worksites, supervising teleop, resets, and safety. Their workaround is better data economics:

• Pre-train wide; post-train right on a small gold set.

• Protect language so demos remain reusable across tasks and robots.

• Plan location entropy (fewer visits, more diversity).

• Reuse legacy/static datasets to subsidize mobile skills.

• Scale supervision with hierarchical prompting and LLM relabeling.

• Use targeted RL when imitation plateaus.

• Add experience-retrieval memory for multi-stage tasks like “put it back where it was.”

How my view evolved

I came in thinking “more data = more general.” I left convinced the bottlenecks are instruction-set design and coverage planning. Bound both, ship earlier, then widen the envelope (fold → laundry; clear table → clean kitchen).

“Scaling from Simulation to Reality”

Both kept us honest. Today’s blocker isn’t clever models; it’s getting reliable skills into messy, revenue-bearing settings. Early scale will include human oversight, CI/CD with simulation, and rollback plans. Synthetic data helps but must be filtered. Expect fit-for-purpose rigs to land before household humanoids; ROI is driven by utilization and intervention rates, not parameter counts.

How I judge vendors now: show interventions/hour, MTBF/MTTR, success vs. #locations, and episodes to add a new SKU. If those curves aren’t on the table, it’s still a demo.

Field notes (quick hits)

• Stop-gradient “wall.” Small architectural change, big effect: keeps instruction-following intact while control learns.

• “Rare but relevant” slices. Tiny amounts of mobile data still moved transfer - curate, don’t hoard.

• Coverage, not luck. Transfer improved steadily with location count - treat data like test coverage.

• Grounded open-ended prompting. LLM relabeling scales breadth; keep prompts tied to state changes, not just fluent text.

Short playbook that I’ve scribbled on my way back

If you’re building:
Freeze a language-faithful VLM, attach a separate Action Expert, and stop-grad the control head. Stand up a gold post-train set and report deltas vs. pre-train. Launch a location-coverage plan and chart transfer. Add experience-retrieval memory for multi-stage tasks. When imitation plateaus, use RL on the tightest failure clusters.

If you’re buying or investing:
Ask for pre+post vs. all-data ablations, instruction-following under shift, location-count curves, and live intervention stats. Review the ops plan: oversight ratio, CI/CD-in-sim, and rollback procedures.

My final synthesis - Assistants plan, robots prove

Mr. Chi’s assistant stack, Professor Finn’s robot stack, and Dr. Kapoo and Mr. Chiu’s deployment forecast rhyme. Assistants plan across tools with personalized rewards; robots prove those plans in the world’s entropy. Both need memory and post-training grounded in real tasks. Moats look like personalization reward models (assistants) and coverage-curated data ops (robots).

What I’ll watch next

1. Assistant → robot handshake becomes a shared convention (planner API + action schema). Wrong if: bespoke glue still dominates by late 2026..?

2. Experience-retrieval memory becomes table-stakes for household-style tasks. Wrong if: we see >95% success on “put-back-where-it-was” with no memory.

3. RL-on-top is the path from ~80% → 95%+. Wrong if: production systems pass 95% with imitation only.

Closing - From capability to commitments

“Generalist” won’t be a single model. It’ll be a disciplined recipe: protect language, budget environment diversity, and treat reliability as a product requirement. That’s how prompts become policies, and how demos graduate into deployments that keep their promises.

Thank you NATEA for bringing together amazing speakers! And I really appreciate all the speakers for sharing their Saturday afternoon to share your view from the frontier.

Smart glasses have already ridden two hype waves: see-through AR in the 2010s and “camera-glasses with an assistant” now. Are we still chasing sci-fi - or is there a practical product wedge that crosses from novelty to daily habit for millions?

TL;DR

The consumer wedge isn’t full 3D; it’s eyes-up utility: translation, wayfinding, capture→memory, quick answers - in <10 seconds, privately, without pulling a phone. MR/VR headsets will keep winning where immersion and 3D matter (training, design). What changed: miniaturized NPUs, better waveguides, and a fashion-first approach proved by Ray-Ban/Meta. The next 2-3 years are about brighter text outdoors, seamless on-device↔phone↔cloud routing, and B2B scale-ups that fund the ecosystem.

1) Decoding the device map

• AR glasses (see-through, heads-up): Look like glasses. Small projector + waveguide or “birdbath” combiner injects text/icons into your real view. Optimized for 2D utility (captions, arrows, menus). Socially acceptable when light and stylish.

• MR headsets (mixed reality): Visor with micro-OLED/LCD displays and video pass-through. Anchors 3D objects with depth/occlusion. Magical for 3D tasks, but bigger/heavier/costlier.

• VR headsets: Fully occluded, synthetic worlds. Best for games, simulation, focused training.

“AI smart glasses” (today’s focus): Lightweight AR-ish frames with camera(s), mics, tiny display, and on-device AI; escalate heavier work to phone/cloud. Not holograms - useful 2D overlays + context-aware assistant.

Product truth: AR glasses excel at glanceable utility and all-day wear; MR wins at rich 3D. Mixing those jobs in one form factor usually yields heavy, hot, unloved hardware.

2) What actually changed (why “now” feels different)

• Miniaturization + on-device AI. Wearable-class SoCs/NPUs can now do wake-word, OCR, short translation, and simple vision locally (sub-watt), while routing complex prompts to phone/cloud. (Think Snapdragon AR-class splits across glasses + phone for ~2–3× on-glass AI perf at lower power.)

• Optics getting practical. Newer waveguides improve in/out coupling and efficiency, pushing toward outdoor-readable text without nuking battery. “Birdbath” variants still win on brightness for some SKUs.

• Fashion meets function. Ray-Ban/Meta showed that industrial design is a feature. Partnerships with Warby Parker / Gentle Monster signal the category is done shipping “tech goggles.”

• Clearer use-case focus. Teams are doing less, better: translation, wayfinding, capture→memory, quick answers. (Not dragons in the living room.)

Human angle worth noting: Knowlab highlights accessibility wins (e.g., AI glasses paired with retinal implants restoring functional vision in studies). Beyond consumer convenience, there’s real social value - live captions for hearing-impaired, scene description for low vision, cognitive assist.

3) Who’s shipping what (and what’s brewing)

The Tech Giant Arms Race is ongoing, and every companies are investing into theri own smart glasses. Four years ago when I was doing my first project on XR devices, it seemed like companies have halted aggressive investment, waiting for consumer tech trendsetters (e.g. Apple) to release and make the market first. But companies never totally stopped - they were always brewing new patents and partnership underneath. It’s exciting to see those finally come to life.

• Meta × Ray-Ban: Category leader in “assistant camera-glasses.” Recent Display model adds a small screen plus neural wristband control. Trying to earn mainstream style + viral utility = volume.

• Apple: Multiple reports point to iPhone-companion glasses targeted for announcement ~2026, ship ~2027: all-day wear, low-power Apple-Watch-class silicon, assistant-first before heavy AR. Strategy rhymes with Vision Pro: seed devs early, ship when comfortable. There aren’t any ‘public & official’ videos yet, so brought one that reviews on most recent leaks.

• Samsung / Google (Android XR): Galaxy XR headset launched; AI glasses in partnership with Warby Parker (mass) and Gentle Monster (fashion), integrated with Gemini. Google gets a second at-bat, this time with style and an AI-first stack.

• Amazon Echo Frames: not AR, but firmly in the smart-glasses lane — and as an international student I basically live on Amazon 😂. They’re audio-first (no display/camera): open-ear audio + Alexa for quick, hands-free tasks. Different wedge from Meta’s camera+display (ambient assistant, zero visuals) that still builds habit and ecosystem pull. But what’s brewing? Amazon is piloting AR glasses (Amelia glasses) for delivery drivers (HUD for navigation, scanning, proof-of-delivery) - enterprise-first, not broadly released yet.

• China plays offense: Alibaba Quark AI Glasses and Xiaomi AI Glasses push aggressive price/feature mixes (real-time translation, payments, livestream), validating consumer demand at scale.

• Snap (2026): Public commitment to consumer AR Specs after >$3B invested across 11 years - betting on see-through lenses, creator-led use.

• MR headsets: Apple Vision Pro, HoloLens 2/Magic Leap 2, Meta Quest family - increasingly pro/work tilted: training, design reviews, remote support, spatial workflows.

Read the room: Today’s divergence is clear - AR glasses aim B2C (eyes-up micro-tasks), while MR/VR aim B2B (immersion-heavy jobs with measurable ROI).

4) Use cases: The jobs to be done

Thesis: PMF likely to land faster in B2B (centralized buyers, immediate ROI, indoor optics) with MR/VR headsets. Consumer demand for AR glasses would follow once weight, outdoor readability, and price clear thresholds - anchored by travel/translation, wayfinding, capture→memory.

Below are the breakdown of use cases that already exist + I think would likely come into reality soon. Also tried to bring back memories from my past projects on XR devices :)

B2C – Everyday, eyes-up utility

Smart glasses will earn their spot on your face not through holograms, but through micro-moments of value that happen dozens of times a day.

1. Capture → Memory
   This isn’t just about taking memorable 3D pictures/videos that you can already do on Apple Vision Pro. This is the “never forget again” layer. When you see something worth remembering - say, where you left your keys, a whiteboard note, or a wine label - the glasses capture that moment with context (“Saved: keys on desk, 10:32 AM”). Later, you can simply ask, “Where did I leave my keys?” and it recalls the visual. It’s not life-logging; it’s human-scale recall, bridging the gap between short-term memory and searchable vision. How far this could go? It’s not just keys that it can remember - things, locations, people’s face and characteristics, and more.

2. Travel & Translation
   You glance at a menu in Tokyo or a metro sign in Paris - the text auto-translates and prices convert instantly (“Ramen Bowl – $8.50”). The detected language appears as a soft badge in your view. This isn’t about showing off AR tricks; it’s reducing friction in unfamiliar contexts, turning every traveler into a confident local without breaking conversational eye contact.

3. Fitness & Outdoors
   Think of it as a heads-up dashboard for movement. While running or cycling, you see pace, heart rate, and calories inline with the road ahead, plus contextual info like your playlist or route cues. No wrist flicks or screen taps - just continuous feedback that keeps your eyes on the path. XR here isn’t about 3D worlds; it’s about flow - keeping you present while still informed.

4. Events & Venues
   In crowded, sensory-heavy spaces like concerts or sports games, smart glasses surface context without distraction. Lyrics, next-song cues, and event info pop up as lightweight overlays, while still letting you enjoy the moment. Imagine your favorite band starting the encore and your glasses quietly reminding you which song is next or where your friends are sitting. It’s the difference between capturing the vibe and missing it while checking your phone.

5. Wayfinding & Local Discovery
   Navigation becomes ambient. Arrows align with the street ahead, and local offers or recommendations surface in context (“Café 120 m → 10 % off latte”). You’re no longer toggling between the world and a map - the city annotates itself. It’s not just directions; it’s eyes-up discovery that blends movement, commerce, and curiosity into one frame.Together, these B2C cases revolve around one theme - micro-tasks in under ten seconds, done without pulling a phone. That’s the true wedge before AR goes mainstream.

[B2B Use cases]

B2B – Immersion with ROI

In the enterprise world, adoption is faster because ROI is clear, optics are indoors, and training costs are high. XR devices already show measurable gains here.

1. Simulation Training
   Picture a new hotel front-desk employee or retail associate wearing lightweight MR glasses. Instead of static manuals, they practice live scenarios - greeting guests, checking reservations, handling edge cases - all guided by an overlayed trainer avatar. The headset tracks eye contact and timing, offering real-time cues (“Step 1 → Greet guest ✓ Step 2 → Check reservation”). It’s learning-by-doing, scaled through simulation. What used to take days of shadowing now happens interactively in minutes.

2. Inventory & Logistics
   In a warehouse or fulfillment center, AR glasses display what’s next - shelf location, quantity, pick confirmation - directly in the worker’s view (“Shelf A3 · 18 units · Next Item →”). No handheld scanners or clipboards; both hands stay free. The system syncs live with ERP data, cutting picking errors and boosting throughput. It’s the quiet side of XR: clear direction, zero confusion.

3. Remote Assist & Maintenance
   When a technician faces a complex machine fault, they can summon an expert instantly. The expert sees what the technician sees through the headset’s camera and can draw annotations directly into the field of view (“Step 3 → Tighten bolt”). This collapses distance - senior engineers no longer need to fly out, and even junior techs can perform advanced repairs safely. The payoff is faster resolution and higher first-time-fix rates.

4. Design Visualization & Prototyping
   Engineers and designers can “pull” digital prototypes into the physical world. A new phone casing or robotic arm appears in mid-air, rendered at real scale and adjustable by hand (“Material: Aluminum · Angle: 32°”). Instead of imagining CAD models on a flat screen, teams walk around and iterate instantly - testing proportions, materials, and ergonomics in context. XR becomes a spatial whiteboard for product imagination.

5. Factory Quality Inspection
   Inspectors see the line through computer vision assistance. The system flags misaligned screws, missing labels, or cosmetic defects in real time (“Defect: Misaligned screw → Fail”). Each check is logged automatically with timestamp and item ID. The glasses act as a second set of eyes - tireless, consistent, and precise - ensuring that every unit meets spec before it ships.

5) Hard constraints, and how teams are attacking them

A) Displays & optics (brightness, efficiency, eyebox)

• Waveguides trending to higher coupling efficiency → fewer nits for same legibility; reflective variants help outdoors.

• Micro-OLED/µLED + better collimation raise brightness without cooking the temple.

• Mechanical: distributed batteries (both temples/nose), slimmer driver boards, using the frame as heat spreader.

B) Meta-optics for cameras/sensors (thinner “eyes”)

• Metasurface lenses compress multi-element stacks into sub-mm “flat” optics - already shipping in depth/eye-tracking modules.

• Expect early wins in gesture/eye tracking and short-range depth (smaller FOV needs), enabling sleeker fronts without bulging camera pods.

C) Vision comfort & eyesight

• Fix the vergence–accommodation mismatch with comfortable virtual focal distance for text (~1.5–4 m), conservative motion, high contrast.

• Prescription integration (lens inserts, 3D-printed Rx with embedded films), electrochromic dimming outdoors, and per-user calibration.

D) On-device AI & routing (the real unlock)

• Tiered compute: glasses NPU handles wake word, OCR, short translation, simple vision; phone/cloud do heavy LLM or multimodal reasoning.

• Product magic is an invisible router that chooses local vs phone vs cloud, keeps latency <~300 ms for common actions, and manages KV-caches/streaming so it feels instant.

Accessibility side note (from Knowlab): integrated AI vision aids - scene description, live captions - plus research using retinal implants + AI glasses shows non-gadget, life-changing impact.

6) Market momentum (signals from the field)

• Meta × Ray-Ban validated style + utility can move units (and eyewear equities). New Display variant and neural wristband push input UX forward.

• Samsung/Google are seeding an Android XR runway with fashion partners (Warby Parker, Gentle Monster) and Gemini on-device.

• Apple is likely to prime the pump with a dev-first reveal ahead of launch, mirroring Vision Pro’s ramp.

• Alibaba, Xiaomi show consumer price/feature elasticity at scale (translation, payments, live-stream) with aggressive weights (~40 g without lenses reported on some SKUs).

• Snap commits to 2026 consumer AR Specs after a decade+ of R&D.

Forecasts (directionally) point to a steep curve: multi-tens of millions of units by decade’s end if the category holds its current trajectory; B2B roll-outs will subsidize the tech curve while consumer SKUs find their wedge.

7) Roadmap: what I think would tangibly change in the next <5 years?

• Outdoor text readability becomes “good enough.” Brighter, higher-contrast overlays; smarter auto-dimming/contrast modes.

• Latency choreography tightens. Common asks (translate this sign, summarize this page, what’s that building?) return answers in ~1–2 s via smart local↔phone↔cloud routing.

• Quiet inputs mature: subtle frame swipes/pinch/eye-affirm beats “talk to your glasses in public.”

• Memory as a feature stabilizes: capture → auto-organize → “find again” by voice works reliably.

• Enterprise scale: four-figure deployments in field service, warehousing, assembly; AEC pilots widen; training becomes a horizontal SKU.

• Consumer wedge hardens around travel/translation + wayfinding + capture→memory, priced to move (and likely carrier-bundled).

• New business models. Especially for MR/VR headsets: As OEMs push hard to create mass replacement of desktops to MR/VR headsets, rental businesses (just like how companies rent laptops to large corporates) may start happening, starting from B2B side.

Signals to watch

• Sub-150 g frames claiming 4–6 h mixed use, explicit “outdoor-readable text.”

• Phone OS releases with deeper handoff APIs for glasses.

• Carrier bundles and retail tie-ups.

• Case studies with hard deltas: first-time-fix↑, picks/hour↑, rework↓.

8) Closing Answer

So - can assistant-first smart glasses move from novelty to habit? Yes, if they embrace constraint. The winners won’t chase full 3D fantasies on a 40-gram frame. They’ll do less, better: instant translation, arrows when you need them, hands-free capture that turns into trustworthy memory, and quick “what’s that?” answers - fast, private, stylish. (TMI: To me it’s interesting to see this trend happening across verticals - just as I explained in my prior article on ‘smaller’ AI models. Techs shifting gear towards mass adoption and scale up tending to go smaller but better.)

Short-term, MR/VR headsets own 3D-heavy work (training, design, remote assist); AR glasses earn the consumer day by nailing eyes-up micro-tasks.

As optics get brighter, NPUs get cheaper, and routing gets invisible, I am convinced that AR glasses trend toward “the next smartphone” - the thing you don’t leave home without - while MR/VR becomes the next laptop - heavier, but where deep work happens.

The bar is simple and ruthless: useful in under 10 seconds, without fuss. Teams that meet it - respecting human factors as much as silicon - will own the face, and the balance sheet that follows.

How did a “routine” DNS issue in one AWS region ripple into banks, games, and government sites - and what designs keep the next hiccup from becoming everyone’s outage?

TL;DR

A DNS misfire for a DynamoDB endpoint in US-EAST-1 triggered synchronized retries and service-chain stalls across thousands of apps. Resilience comes from multi-region control planes, DNS canaries/rollback, sane client retries + circuit breakers, and practiced failovers. Outages happen; internet-wide cascades don’t have to.

Cheat Sheet before Diving In

• DNS: name → IP. If it’s wrong/slow, everything above it wobbles.

• Control plane: orchestration brain - keep multi-region and partitioned.

• US-EAST-1: AWS’s gravity well - minimize hard dependencies.

• Retry storms: synchronized client retries that amplify outages—use backoff + jitter.

• Active-active: multiple regions serving at once; no cold glass to break.

• TTL: DNS cache time; balance agility vs stability.

Decode the Basics

The cloud in three lines. You rent compute, storage, and platforms over the internet instead of buying servers. IaaS = raw machines; PaaS = managed building blocks (databases, queues, identity); SaaS = finished apps. The draw: speed, elasticity, and managed ops.

Two planes, two risks. The control plane (the brain) creates/updates resources and enforces identity/policy. The data plane moves live traffic. If “global” control services quietly anchor to one region, a small fault can jam a lot.

DNS is the phonebook. It translates names (e.g., an API endpoint) to IPs. If DNS lies or times out, clients can’t find the door. Cache settings (TTLs) and client behavior decide whether you recover quickly—or stampede the network.

One-screen mental model

What Just Happened (Oct 20, 2025)

• Where: US-EAST-1 (N. Virginia), AWS’s busiest historical region.

• Trigger: A DNS resolution issue on the DynamoDB API endpoint after an update.

• Blast radius: >1,000 companies impacted; consumer apps, games, banks, and gov portals showed failures.

• Why it spread: US-EAST-1 gravity + service chaining + synchronized retries = a mundane fault turned very public.

• Recovery: DNS was mitigated the same day; lingering queues/backlogs cleared afterward.

• Note: No signs of a cyberattack, this was configuration/infra behavior. And yes, “it’s always DNS” is more pattern than joke.

Product & Operations Lens: what actually reduces blast radius

1. Multi-region by design. Treat identity, metadata, and routing as multi-region products (active-active for reads; queue + replay for writes). Avoid single-region anchors.

2. DNS with brakes. Dual/secondary DNS for crown-jewel names, staged/canary changes with instant rollback, health-checked CNAMEs, thoughtful TTLs. Treat DNS edits like code.

3. Clients are part of resilience. Ship exponential backoff + jitter, circuit breakers, idempotent writes, and bounded queues. Fail fast with clear UX instead of hot-looping retries.

4. Practice the failure. Run DNS game-days and region-evac drills during low traffic. Simulate “endpoint unreachable.” Measure real RTO/RPO - not slideware.

What Providers Are Doing (and what you can lean on)

• Partitioned control planes / cell architectures to contain faults.

• Safer DNS pipelines (canaries, health checks, fast rollback).

• Stronger SDK defaults (built-in backoff, circuit breakers, idempotency).

• Decoupling “global” services from US-EAST-1.

• Security & posture tooling (policy guardrails, config drift alerts, retry-storm detection).

• Dual-DNS / edge partnerships (Route 53 + Cloudflare/NS1/Akamai) for split authority.

Goal: localized degrade, not headlines.

Market & Strategy Lens

• Concentration risk ≠ cloud is bad. It’s a design-for-failure problem.

• Multi-cloud helps only if shared dependencies (DNS, identity, data) are detangled; otherwise you just doubled contracts.

• Regulatory push (EU/UK) will ask banks/gov for evidence of DNS/identity/data redundancy and region independence.

• Economics: Compare $/minute downtime vs $/month resilience. DNS canaries, SDK backoff, and dual-DNS are cheap relative to lost revenue and trust.

Signals to Watch

• Specific post-mortems: DNS canary/rollback SLAs; region-partitioned control planes.

• SDK updates you can feel: default backoff/jitter, circuit breakers, idempotent calls.

• Customer detangling: dual-DNS adoption; published region-evac runbooks; moving IAM/metadata off single-region anchors.

• Tooling uptick: DNS change-management/canary tools; multi-region data products.

• Regulatory guidance on cloud resilience audits that include name-resolution SLOs.

What I’m Still Unsure About

• How much “global” control still transits US-EAST-1 under the hood - and the pace of unwinding it.

• Whether customers will fund active-active + dual-DNS after the news cycle fades.

• How far providers push SDK-level defaults to curb client-side storms.

Closing Answer

Too many “global” paths still breathe through US-EAST-1, and naming is oxygen. When DNS for a core endpoint failed, clients all retried together, chains jammed, and the blast radius grew. The fix isn’t glamorous: multi-region control planes, DNS isolation with canaries, client backoff/circuit breakers, and regular game-days. You can’t stop hiccups, but you can keep them from becoming headlines.

TL;DR: The spark is here; the wildfire starts on factory floors. As shared-space safety matures, reliability improves, and costs keep sliding, humanoids scale from narrow tasks to wider shifts - less a viral jump, more a compounding curve.

Why Humanoids, Why Now?

Last time I wrote about a specific product within humanoids, but this time I’ll zoom out to explain about the humanoids overall.

To the 20th century, robots were machines behind fences; to the 21st, they may stand beside us.

Humanoid robots are no longer just science fiction or viral demo material. They’re being piloted in factories, warehouses, and even hospitals, aiming to fill labor gaps and automate tasks in spaces built for humans. Unlike the home-focused Figure 03, which spotlights imitation learning and consumer chores, this article zooms out: What makes humanoids possible, what are their industrial frontiers, and what will it take for them to scale?

What Is a Humanoid Robot, Really?

A humanoid robot is a machine designed to operate in human environments, with a body plan (two arms, two legs, a head) that lets it use our tools, open our doors, and navigate our world. But the real story is how these robots combine mechanical engineering, advanced sensors, and AI-driven control to move, perceive, and act with increasing autonomy.

But humanoid robots are not all the same - most real-world models fall into four broad categories, each with distinct roles and interaction styles:

This 4-part categorization helps clarify why robots like Figure 03 feel so different from, say, a logistics robot rolling through an assembly line: each is built for a different spectrum of engagement and capability.

The Anatomy of a Modern Humanoid

• Skeleton & Actuators: Lightweight alloys and high-torque motors mimic bones and muscles, enabling walking, lifting, and balancing.

• Sensors: Cameras, LiDAR, tactile pads, and inertial sensors provide vision, touch, and balance - crucial for safe, adaptive movement.

• AI “Brain”: Onboard computers run perception, planning, and control algorithms, often powered by the same chips that drive generative AI.h-density batteries and efficient power management are essential for multi-hour operation.

The Four Industrial Bridges

While home robots like Figure 03 focus on learning from demonstration, industrial humanoids face a different set of hurdles:

1. Safety in Shared Spaces

   • Robots must work alongside people without cages, requiring robust perception, fast reflexes, and compliance with evolving safety standards.

2. Uptime and Reliability

   • Industrial buyers demand robots that can run for hours, self-diagnose faults, and recover from stumbles - no babysitting allowed.

3. Dexterity and Versatility

   • From handling car parts to sorting packages, humanoids need hands and arms that can adapt to a wide range of objects and tasks.

4. Cost Down, Scale Up

   • To move beyond pilots, costs must drop from six figures to the price of a car. This means modular designs, mass manufacturing, and a robust supply chain.

The Tech Stack: What’s Important and New?

• Actuators: Next-gen motors and gearboxes deliver human-like strength and speed, with fewer parts and less maintenance.

• Perception: Multi-modal sensor fusion (vision, touch, force) enables robots to “see” and “feel” their environment in real time.

• AI Control: Large models trained in simulation and real-world data allow robots to generalize across new tasks and settings - crucial for industrial flexibility.

• Cloud & Edge: Many robots now blend onboard processing with cloud-based updates, letting them learn from each other and improve over time.

Where Are Humanoids Working First?

Humanoid robots are found across sectors, but many pilots are found in manufacturing and logistics, mainly due to the fact that it is the only industry at this point that can justify the high initial investment into product & installation. Looking more specifically into some of the sectors, below are the specific sub sectors that are increasingly trying out humanoids:

• Automotive Plants: Robots handle repetitive, ergonomic tasks—moving parts, loading machines, and even quality checks.

• Warehouses: Early deployments focus on picking, packing, and moving goods in spaces designed for people, not conveyor belts.

• Healthcare & Labs: Some humanoids assist with patient mobility or repetitive lab work, where adaptability and safety are paramount.

Emerging Types of Humanoids

If Figure 03 (the one I introduced in my last product-decoded article) is one of the more approachable, home-oriented humanoids you’ll soon find around you, there are more ambitious types on the horizon - each opening new markets and technical frontiers.

• Morphing and Multi-Modal Humanoids:
  This is the robot that made me write this article. A member from my Stanford Student Robotic’s Club uploaded this video on our discord, saying that our drones should be able to deliver to the doorstep via transformative robot modules.

  Caltech’s X1 system merges a bipedal humanoid (Unitree G1) with a drone-like robot (M4) that can fly, drive, and roll. This hybrid can walk, deploy a flying scout, and traverse obstacles - showing how modular, multi-modal robots could tackle logistics, disaster response, and exploration in ways single-mode robots cannot.

• Industrial-Grade Heavy-Lift Humanoids:
  Robots like Tesla’s Optimus Gen 2 and NEURA Robotics’ 4NE-1 are engineered for manufacturing, logistics, and construction, with reinforced frames and high payload capacity. These are designed for demanding environments and tasks that require both strength and adaptability.​

• Swarm-Enabled and Collaborative Humanoids:
  Some startups are developing humanoids that work in coordinated teams, sharing data and dividing tasks - potentially transforming warehouse automation and large-scale logistics.

• Specialized Healthcare and Research Humanoids:
  Models like Fourier Intelligence’s GR-1 are tailored for patient mobility, rehabilitation, and lab work, while others serve as platforms for advanced AI and human-robot interaction research.

For venture capital, these emerging types represent not just incremental improvements, but new business models and market opportunities - modularity, collaboration, and autonomy are the next big bets.

The Business Case: Why Industry Cares

• Labor Shortages: Aging populations and tight labor markets make automation attractive, especially for dull, dirty, or dangerous jobs.

• Flexibility: Unlike fixed automation, humanoids can be re-tasked for new workflows without expensive retooling.

• ROI: As costs fall and reliability rises, the break-even point for humanoids in logistics and manufacturing is approaching fast.

Roadblocks to Mass Adoption

Despite the promise, mass adoption of humanoids faces deep technology and business roadblocks:

• Dexterity and Manipulation:
  Most humanoids still lack the fine motor skills and adaptive control needed for unstructured environments. Human-level manipulation remains a massive engineering challenge, limiting real-world deployment to repetitive or highly structured tasks.

• Energy Efficiency and Runtime:
  Multi-jointed limbs and balancing systems consume significant power, resulting in short runtimes (often 2–4 hours per charge). This restricts use to short shifts or requires frequent battery swaps, making them less practical than task-specific robots.

• Speed and Payload:
  For safety and balance, humanoids move cautiously and are slower than both humans and industrial robots. Their payload capacity is also limited, making them unsuitable for high-speed, heavy-duty environments.

• Cost and Scalability:
  Building a humanoid robot can cost anywhere from $16,000 to $300,000+. The absence of a mature supply chain means companies often have to design and manufacture components from scratch. Achieving economies of scale is critical to bringing costs down to the $20,000–$50,000 range needed for mass adoption.

• Regulation and Liability:
  Regulatory frameworks for humanoids are lagging behind technology. Certification for workplace safety, liability in mixed human-robot environments, and ethical considerations are all unresolved, slowing large-scale deployment.

• Social and Political Resistance:
  Humanoids are designed to take over jobs currently done by humans, which can lead to resistance from workers, unions, and policymakers. Public acceptance and trust will be critical for adoption.

• Market Hype and Investor Pressure:
  Fast funding cycles and aggressive investor expectations can push companies to over-promise and under-deliver, risking market disappointment and a correction similar to the autonomous vehicle sector.

How the Industry Is Tackling These Barriers

The field is attacking these challenges with targeted innovation and collaboration:

• Dexterity:
  Labs and companies are developing new actuators, soft robotics, and modular hands to mimic human flexibility. Imitation learning - robots learning by watching humans - improves generalization and reduces the need for hand-coded routines.

• Energy and Runtime:
  Higher-density batteries, swappable packs, and lightweight materials are being integrated into new models. Fast-charging infrastructure and energy-efficient actuators are extending operational hours.​

• Cost and Scale:
  Modular, standardized designs and mass production techniques, often borrowed from the automotive sector, are driving down costs. Open-source platforms and shared supply chains are enabling broader participation and faster iteration.

• Software and Reliability:
  Large, multimodal AI models trained on both simulation and real-world data are making robots more robust. Self-correcting algorithms and error recovery routines are being built in to handle unexpected situations.​

• Regulation and Safety:
  Industry groups and standards bodies are piloting new safety protocols, including real-time force limits and certified fall recovery. Companies are working with regulators to gather safety data and refine compliance pathways.

• Social Acceptance:
  User-friendly interfaces, transparent privacy policies, and collaborative training programs are being piloted to build trust and ease workforce integration.​

The next breakthroughs will likely come from modular hardware, smarter AI, and real-world pilots that prove reliability, safety, and ROI at scale.

The Bottom Line

Humanoid robots are poised to become the next industrial revolutionaries - not by mimicking every human gesture, but by automating the physical work that keeps our factories, warehouses, and hospitals running. The winners will be those who cross the four industrial bridges: safety, uptime, dexterity, and cost.

The age of the embodied AI colleague is just beginning. The question is not if, but how fast - and who will lead the charge.

TL;DR: “Bigger = smarter” is giving way to “smarter per watt, per dollar, per millisecond.” Tricks like mixture-of-experts, distillation, and quantization - plus on-device chips - let smaller models feel fast, cheap, and (for many jobs) just as good.

The end of the size war

Two or three years ago, the leaderboard felt simple: whoever had the most parameters seemed to have the best model. We spoke in billions like it was casual - 7B, 70B, 175B. OpenAI’s GPT-3 set the tone; GPT-4 raised it. Meta, Google, Anthropic, and others chased the same horizon. Scale looked like destiny.

And then the bill arrived.

Training a frontier model isn’t a one-off indulgence. It’s a repeating nine-figure commitment, with power use big enough to light a small city — and rising per-query costs when you deploy the model. Reported numbers for the latest giants show steep increases in API cost vs. earlier models, alongside growing concern that scaling alone is hitting diminishing returns.

Even if you can afford it, users still judge you on something brutally simple: latency. If an answer takes two seconds instead of 300 milliseconds, it feels worse - no matter how clever it is.

That’s when the conversation shifted from How big? to How efficient?

What changed: efficiency beats mass

A smaller model doesn’t have to be a dumber one. Modern training and inference squeeze more work out of fewer parameters, and map cleanly to today’s hardware.

• Mixture-of-Experts (MoE): Route each token to a couple of specialists, not the whole brain. Like pinging the right teammate instead of dragging the entire company into a meeting.

• Distillation: A compact “student” copies a large “teacher’s” habits - how it balances options, how it handles tricky prompts - without carrying every neuron the teacher has.

• Quantization: Store numbers in fewer bits (8-bit, even 4-bit). That slashes memory traffic - the quiet killer of inference cost - and boosts throughput on modern accelerators.

Add pruning (drop low-value connections), smarter decoding (draft-and-verify, prefix reuse), and tidier memory handling (KV-caches that don’t thrash), and you get a surprise: small models that feel fast, cheap, and, for many tasks, good enough.

You can see the industry leaning in. OpenAI’s GPT-4o mini launched at $0.15 per million input tokens and $0.60 per million output - more than 60% cheaper than GPT-3.5 Turbo - a clear signal that efficiency matters as much as peak IQ. Meta’s Llama 3.1 shows how far algorithmic efficiency has come: its largest variant competes with top proprietary models at lower runtime cost, and it’s open.

Why on-device matters

Running a model on your phone or laptop used to be a parlor trick. Now it’s a business decision. Device-class NPUs (neural processing units) make local inference practical. The payoff is threefold:

• Speed: No round-trip to the cloud; sub-second replies become the default.

• Privacy: Sensitive inputs can stay on your device.

• Cost control: Not every tap hits a cloud GPU.

Apple planted a flag with “Apple Intelligence.” Qualcomm, AMD, and Intel are shipping chips with AI blocks for everyday apps. Once people feel zero-lag assistants and private summarization, it’s hard to go back.

The new architecture: fleets, not monoliths

In the scale era, teams tried to do everything with one colossal model. In the efficiency era, teams run fleets:

• a tiny router that figures out what you’re asking,

• a fast generalist (think 3–9B parameters) for common queries,

• and a few specialists - vision, code, deeper reasoning - that spin up only when needed.

You don’t bring an orchestra to play a doorbell chime. You ring the bell. If it turns into a symphony, then you hire the strings.

This is where retrieval earns its keep. Instead of cramming the entire internet into weights, you fetch what’s needed - docs, tables, recent emails - and let a smaller core reason over fresh context. In practice, memory bandwidth and search (not raw parameter count) become your real constraints.

What “good” looks like in products

If you’re shipping, a few truths tend to hold:

• Latency wins hearts. Sub-300 ms replies feel magical.

• Quality-per-token beats peak scores. A distilled ~9B with retrieval and guardrails often beats a giant model that costs 10× and responds slower.

• Privacy is a feature. Run local for personal data; escalate to cloud only when you truly need the bigger brain.

• The ops bill is a design input. Quantization, fused kernels, small prompts, and tight KV-cache handling aren’t “optimizations”; they’re part of the product.

Failure modes have changed, too. It’s common to see a tiny model wrapped in a giant prompt, saving on model size but overspending on context. Or a pipeline that’s compute-fast but I/O-bound. The fixes are mostly discipline: slimmer prompts, smarter retrieval, and kernels that keep data close to compute.

How the business map shifts

Value moves when constraints move. Shrinking models and growing NPUs shift power in a few places:

• Cloud → Edge balance. You still need the cloud for heavy training and some inference, but more experiences happen locally. That cuts variable cloud cost and enables “always-on” features without scary bills.

• Model providers → solution providers. The moat isn’t “biggest model” anymore; it’s best model for the job, packaged with retrieval, tools, and guardrails.

• Hardware differentiation. Device makers who pair capable NPUs with clean developer stacks will pull ahead. The best apps will feel “instant” because they’re local.

• Metrics that matter. Accuracy still matters, but operators care just as much about latency, tokens-per-dollar, and energy-per-query. That’s why cheaper, capable “mini” and open models are gaining steam - they’re usable and affordable at scale.

Why “smaller” now: two simple realities

1) Diminishing returns from brute-force scaling.
Recent reports on the newest giants show incremental quality gains at much higher compute and energy, plus fragile behavior that comes with complex routing. In short: scaling still helps, just not enough to justify the cost in many use cases.

2) Better outcomes from “doing more with less.”
Smaller, purpose-built models can match (or surpass) larger ones on targeted tasks when trained on focused data, and they’re faster to train, deploy, and run, often on local devices. That’s good for product teams and the planet.

Where this goes next

Expect routing and sparsity to get smarter; expect quantization to be the default, not an exotic trick. Expect more apps to start local and escalate only when necessary. And expect “model fleets” to become normal infrastructure, not bespoke wizardry.

There are open questions. How far can a small model go, even with great retrieval, before you truly need frontier-level depth? Will on-device ecosystems converge (easy for developers) or fragment (porting pain)? As agents take on longer, tool-heavy workflows, does the pendulum swing back toward bigger centralized brains - or do routers and specialists keep the upper hand?

The answer, finally

This isn’t a detour. It’s a structural turn. Frontier models still matter for pushing the ceiling, but most of the daily value - what users feel and businesses pay for - will be delivered by smaller models that are fast, local, and paired with the right data.

You don’t just save money. You ship more features, to more people, in more places. The steady drop in “mini” model prices and the rise of credible open options are the tells.

TL;DR

• What shipped: Figure unveiled Figure 03, a humanoid doing multi-step chores (tidy toys, rinse & load, laundry + detergent) while running on an in-house robot brain (no longer piggy-backing on ChatGPT).

• The big shift: Instead of “talk smart,” Figure is going after “move smart”- training robots by watching humans (imitation learning) so you teach by doing, not by programming.

• Why this matters: This could be the ChatGPT moment for robots: the day regular people become robot “teachers,” the market moves from factory lines to living rooms.

• Reality check: Motions are slow, scenes look controlled, and rivals (Optimus/Atlas) exist. But the direction of travel - from expert code → everyday demonstration - creates the flywheel.

• What to watch: the data engine (how much/high-quality demo data), policy architecture (how the brain learns long, smooth actions), tactile sensing & safety, and real-home benchmarks that generalize beyond the lab.

Quick rewind: “Goodbye to the borrowed brain”

Earlier demos used a borrowed brain (ChatGPT-assisted behaviors). With Figure 03, the company says the brain is now in-house, trained end-to-end specifically for seeing → deciding → moving. The timeline is fast (months, not years), which suggests this wasn’t a scramble post-split; it was a planned blueprint.

Why change the brain? Because charming conversation doesn’t wash dishes. Household value = safe, repeatable motion. That’s the product bet.

Product decoded: how a home robot actually “thinks”

Below is a simplified “stack” you can keep in mind while watching any home-robot demo:

1. Sensing

   • Eyes: multiple cameras (likely RGB + depth).

   • Hands/skin: tactile pads to feel slip/pressure (so plates don’t drop and detergents don’t spill).

   • Inner ear: IMUs for balance; joint sensors for knowing where limbs are.

2. Perception (“What am I seeing?”)

   • Detects objects (plate, cup, knob), places (sink, rack), affordances (“this part is graspable”), and states (door closed/open, water on/off).

3. Policy (“How do I move?”)

   • The brain learns to map camera/touch inputs directly to arm/hand trajectories.

   • In practice this looks like imitation learning: humans demonstrate tasks; the robot learns smooth, long-horizon behavior (e.g., through transformer or diffusion policies known to produce stable motions).

4. Planner (“What’s the next small goal?”)

   • Strings skills into routines: pick plate → rinse → load → close door.

   • Handles retries (slipped plate? regrasp).

5. Control & safety

   • Speed caps, force limits, stop-on-contact, and geofences so the robot moves carefully around people and fragile stuff.

The robot learns to see like a person, move like an apprentice, and follow a checklist to finish the chore without breaking anything.

Why imitation learning is the unlock

Historically, robots repeated motions that an expert coded. To change the task, you hired another expert.

Imitation learning flips it: you show the task; the robot learns the pattern. That means:

• Teaching scales: Any competent adult can become a robot “teacher.”

• Upgrades are data: Performance improves by collecting more/better demonstrations, not rewriting code.

• Generalization: Training across many homes (different dishwashers, knobs, lighting) makes the brain robust, just like how humans learn.

Analogy: ChatGPT moved AI from “build a model” to “just type.”
Figure wants robots to move from “write motion code” to “just show.”

What the demo showed - and what it didn’t

Showed:

• Multi-step sequences: find → grasp → manipulate → place → start machine.

• Hand articulations that look gentle (no crushed plates).

• A run without obvious teleoperation (at least in the published cut).

Didn’t show (yet):

• Speed near human level (today it’s cautious).

• Edge-case chaos (crowded sinks, unfamiliar detergent caps, small sponges, wet/slippery surfaces).

• Recovery from mistakes (mis-grasp, soap spill).

• Long runs (hours of mixed chores), or family safety with kids/pets wandering in and out.

That’s fine for a public first look - but it defines the next milestones.

Data is the new “robot oil”

For action learning, the moat is a data engine, not just a motor:

• Where data comes from:

  • Teleoperation rigs (humans drive arms in VR or with exoskeleton gloves).

  • In-home pilots (privacy-aware capture of household tasks).

  • Synthetic/augmentation (camera angles, lighting, distractors).

• What “good” data means:

  • Coverage: many layouts and brands (Bosch, LG, Samsung dishwashers, different detergent bottles).

  • Tactile variety: slick plates vs. textured mugs; soft clothing vs. heavy towels.

  • Long sequences: so the robot learns persistence (open, place, close, press).

• Why it compounds:

  • More homes → fewer surprises → fewer failure modes → faster and cheaper chores.

Speed vs. safety: how this actually improves

Today’s careful pace is normal. Speed comes from four levers:

1. Confidence: as the brain sees more homes, it hesitates less.

2. Tactile + vision fusion: hands that feel reduce re-tries.

3. Trajectory quality: better planners make fewer stop-start micro-moves.

4. Hardware headroom: most arms can move far faster than demos show; software uncaps gradually, gated by safety.

Expect step-wise unlocks (e.g., a “kitchen beta” that’s deliberately slower around glass, faster around laundry).

Where Figure 03 could beat - or lag - rivals

Versus Tesla Optimus

• Tesla’s edge: enormous data operations, in-house silicon, and a factory playground (repeatable tasks).

• Figure’s angle: prioritize household generalization and imitation UX (teach by doing) earlier.

• Watch: who shows transfer first - “new house, new dishwasher, still works” at acceptable speed.

Versus Boston Dynamics (Atlas)

• BD’s edge: athletic mobility and world-class hardware/control.

• Figure’s angle: chore competence over parkour; hands & dishwashers over jumps and vaults.

• Watch: hand dexterity and tactile progress (opening detergent caps is harder than it looks).

What needs to be true for product-market fit (PMF) at home

• Reliability: ≥95% success on common chores; low oops rate (breakage, spills).

• Safety: predictable force limits and fast stop; child/pet awareness.

• Speed: near human for routine steps; slower for delicate ones is okay.

• Economics: cost of the robot + service beats hiring for the target buyer (or delivers unique value like independence for seniors).

• Delight: setup feels like an appliance, not a research project; teaching a new routine is minutes, not hours.

Skill store: how features might ship

Think in skills (each taught by demos, upgraded by data):

• Kitchen set: rinse & rack, cutlery sort, wipe, trash, recycling.

• Laundry set: load, detergent, run cycle, transfer, fold basics.

• Tidying set: find toys, sort to bins, stack books.

• Errands set (later): bring water/snacks, door answer, basic fetch.

Each skill advertises: success rate, typical time, safe zones, and known caveats (e.g., “works with front-loaders only”).

A non-engineer’s guide to evaluating the next demo

When the next Figure/Optimus video drops, score it on these five:

1. Generalization: new rooms or props vs. the same studio.

2. Contact quality: smooth grasps, no plate clatter, firm but gentle pushes.

3. Recovery: visible retries when things go off-plan—did it fix itself?

4. Pacing: fewer micro-pauses; more continuous motion.

5. End-to-end task close: did it actually finish (door closed, cycle started)?

Risks & open questions

• Data privacy: in-home data is precious - opt-in, on-device processing, and clear deletion are table stakes.

• Edge-case explosion: homes are unpredictable; the long tail is the hardest part.

• Maintenance: hands/joints wear; who services and how often?

• False sense of competence: polished cuts can mask supervision - ask for continuous runs and third-party evals.

Why this still feels like a “ChatGPT moment”

ChatGPT wasn’t perfect; it proved possibility. Figure 03 does the same for action: it reframes robots as things you teach, not things you program. If that unlocks even a narrow set of chores first - and those skills improve with data - we won’t just get a cool demo. We’ll get the first real consumer robot category since the vacuum.

What I’m watching next (3 signals)

1. Benchmarks that matter: standardized household task suites (multiple homes, brands, lighting) with third-party scoring.

2. Brain disclosures: even short notes on policy type (diffusion vs transformer), tactile use, and memory show technical maturity.

3. Speed unlocks with safety: transparent reports on incidents per hour, plate break rate, and how software limits relax over time.

Closing thought

We don’t need perfect; we need proven possible. Figure 03 shows a credible path: teach by doing. If the company can turn that into a repeatable data engine and a safety-first skill store, it won’t just move robots out of factories - it will move them into the rhythm of everyday life.

Appendix

• Imitation learning: robots learn by watching human demos; you show, they copy.

• Policy: the learned “brain” that turns camera/touch into arm/hand motion.

• Generalization: works in new, unseen homes without reprogramming.

• Tactile sensing: “robot fingertips” that feel slip/pressure.

• Recovery behavior: how the robot fixes mistakes mid-task.

TL;DR: OpenAI’s DevDay pushed three big ideas:

1. ChatGPT as an operating system: apps run inside the chat, invoked by intent, not icons.

2. Agent creation for everyone: a no-code “Agent Kit” that assembles useful agents like Lego.

3. Media as programmable: Sora 2 turns sketches and phone clips into cinematic assets; voice becomes the default interface.

What they showed

• Apps inside ChatGPT (Apps SDK).
  Demos stitched Coursera, Canva, and Zillow directly into the chat. You don’t “open” apps; you describe what you want. ChatGPT routes your intent to the right app, pins a video, edits a design, or shows housing inventory - all without leaving the chat window. It’s not just integration; it’s co-working: the model and the app share context in the same space.

• Build agents in minutes (Agent Kit).
  On stage, a developer assembled an agent for the DevDay website with drag-and-drop blocks: a classifier, session-data connector, some conditional logic, guardrails, and UI widgets—no code. The agent went live on the site and answered questions (“Which session for learning agent building?” → “11:15 a.m.”).

• New model + new modalities.
  OpenAI announced GPT-5 Pro (positioned as its most capable model yet), Real-Time Mini (the voice model at ~70% cheaper while keeping quality close), and the Sora 2 API for stateful, directed video. Sora 2 can expand a phone clip into a cinematic wide shot, or turn designer sketches into 3D product mockups (a demo built with Mattel).

Strategy: Why this matters

• From icons to intents.
  Smartphones trained us to tap icons. ChatGPT is training us to declare intentions (“teach me ML,” “make a poster,” “show Pittsburgh listings”). When the intent is the starting point, the operating system is the conversation - and distribution shifts to who owns that conversational surface.

• Lock-in 2.0 (the “AI OS” moat).
  If users stay in one conversational surface for learning, designing, shopping, and booking, that surface becomes the new home screen. The more third-party apps wire in, the stronger the gravity and the harder it is to leave. This echoes Apple’s ecosystem lock-in, but at the workflow level instead of the device level.

• Agents become the new “apps.”
  The Agent Kit reframes “an app” as a directed workflow (classify → fetch → constrain → render) composed by anyone. Expect an explosion of micro-agents pinned to websites, docs, and support portals. The battle will move from who can code to who can design guardrailed workflows that are trustworthy and fast.

• Voice as the default interface.
  With Real-Time Mini, OpenAI is pushing cost down on live conversational models. When voice becomes the cheapest frictionless path, usage frequency climbs and multi-turn context deepens - another flywheel for platform lock-in.

• Media becomes programmable.
  Sora 2 treats video like a stateful object you can direct. That compresses time-to-asset from weeks to minutes and blurs the line between concept, storyboard, and render. The winners will be those who chain Sora-like tools into product pipelines (marketing, industrial design, training content).

What wasn’t said (and still matters)

• The device question.
  Jony Ive and Sam Altman discussed their AI device - no reveals. Hints: many concept variants, human-centered philosophy (“joy,” “connection”), speculation of no screen, context-aware hardware. Target: late 2026. Translation: they’re optimizing for how it feels to live with, not specs. If the AI OS lives in the conversation, the hardware’s job is ambient capture, presence, and gentle delivery - not app grids.

What to watch next

1. Developer economics: How will Apps SDK and agents monetize - usage rev-share, marketplace listings, or enterprise bundles?

2. Trust & safety by design: Guardrails were in the demo. Expect compliance, redaction, and auditability to become standard “features” of agent building.

3. Workflow incumbents: Design suites, CRMs, and vertical SaaS will choose between deep embedding vs. building their own agentic surfaces.

4. Default distribution: Whoever becomes the “first conversational stop” for common tasks (learn, plan, buy, fix) will accumulate the most valuable context graph.

Practical playbook for Founder/PMs

• Ship one high-value agent, not ten features. Pick a painful workflow and compose an agent with clear guardrails and measurable success criteria (time saved, error rate).

• Exploit context hand-off. Use Apps SDK-style integrations to accept state from ChatGPT (user profile, current intent) and return structured results ready for the next step.

• Design for voice first. Make sure your agent’s prompts and outputs degrade gracefully from voice ↔ text ↔ UI, without losing state.

A last word on “AI building AI”

Altman framed this as moving from a lone developer writing code to AI teams collaborating. The shift is not just faster coding; it’s who orchestrates workflows. If the OS is a conversation, the new “programming language” is intent, constraints, and policy - assembled by anyone who can think in systems.

• CUDA is the “operating manual” that lets a GPU act like a huge team of helpers doing tiny tasks at the same time. Most AI software quietly depends on it.

• What’s new: a tile-based way of writing code (think “work on this small chunk, repeat”), a cleaner one-toolkit story for servers and devices, better math libraries, and easier Python.

• New chips (Blackwell) + tiny numbers (FP4/NVFP4) = more work per dollar and per watt for AI inference—if your model operators are supported.

• Why NVIDIA still wins: not just fast chips, but mature libraries, tools, examples, and a giant talent base that have compounding effects.

• What could change: rising portability efforts (UXL/oneAPI), AMD ROCm, and code-conversion tools are improving. Track them, but switch only when your economics say so.

Tiny glossary before diving in (10 seconds each)

• CUDA: The software layer that tells thousands of GPU helpers what to do.

• Tile-based model: Work on a small chunk → repeat everywhere.

• Tensor Core: A special math unit for AI’s matrix math.

• FP4 / NVFP4: Extra-small numbers that let more data flow through without wrecking accuracy.

• Operator coverage: Whether your model’s steps use the fast libraries or slow fallbacks.

• Nsight: NVIDIA’s performance magnifying glass.

A kitchen, not a single chef

Imagine running a restaurant. A CPU is one brilliant chef cooking dishes one by one. A GPU is a kitchen with thousands of line cooks chopping, stirring, and plating in parallel.
CUDA is the binder of recipes and checklists that tells all those line cooks exactly what to do, in what order, and how to share the tools and pans without colliding.

This is why GPUs crush tasks like image recognition or LLMs: lots of similar steps repeated over and over is where a big team shines.

What actually changed in Aug 2025 (and why you should care)

Think of three upgrades:

1) “Tiles” → a friendlier way to program the kitchen

Old CUDA asked you to micromanage workers and storage (who stirs when, which shelf holds onions). The tile-based model says:

“Work on this small square of the job, then repeat everywhere.”
CUDA’s compiler now handles much of the fussy scheduling.

Result: faster developer cycles, fewer foot-guns, and performance that’s closer to what the hardware can really do.

2) One toolkit across servers and devices

Whether you deploy on big servers or on smaller Arm-based devices (Jetson/edge boxes), it now feels like the same toolkit. That reduces “it worked on the DGX but broke on the robot” moments.

3) Python is no longer second-class

You can stay in Python for most of your work (via CUDA Python and NV Math for Python) and only drop into low-level code for the truly hot parts.

Translation: quicker prototypes, less glue code, happier teams.

Why the newest chips matter

New NVIDIA chips (the Blackwell family) unlock two practical things for AI inference:

1. Smaller numbers that still work (FP4 / NVFP4).
   Think of this as zipping your data so it takes less space and moves faster. With the right calibration, models keep accuracy while your throughput jumps.

2. Better “kitchen logistics.”
   Blackwell adds tricks that keep the “line cooks” busy without running back and forth to the pantry:

• Two teams can work as one on big batches (higher utilization).

• There’s a small private shelf next to the stove for ingredients used every few seconds (less back-and-forth to the main storage).

• Wider scoops move more data in one go (fewer trips).

Net effect: For the same quality target, you often need fewer GPUs or can serve more requests per second.

How teams actually build on CUDA (a 4-step loop)

Whether someone’s doing vision, LLMs, or simulations, this is how it generally works:

1. Start with the libraries (fastest wins).

   • Math: cuBLAS/cuSOLVER/cuTENSOR/cuFFT

   • Deep learning: cuDNN (training), TensorRT (fast inference)

   • Python access: NV Math for Python
     These are heavily optimized and usually beat custom code.

2. Profile before you tinker.
   Use Nsight tools to see what’s slow: is it waiting on memory, not using the GPU fully, or tripping on data layout?

3. Turn the big knobs first.

   • Precision: FP16 → FP8 → FP4 where supported.

   • Operator coverage: make sure your model ops map to fast, fused kernels (green paths) rather than slow fallbacks (gray paths).

   • Data layout & fusion: keep data in the right shape and combine steps so you touch slow memory less.

4. Write custom kernels last.
   If a crucial operation still isn’t covered or fast enough, start with the tile approach. Only drop to the old low-level style when you must.

The few knobs that move real money

• Precision: Smaller formats (down to FP4) = more work per dollar - as long as accuracy stays acceptable.

• Operator coverage: If TensorRT/cuDNN fully support your model, you fly. If they don’t, you pay.

• Memory locality: Keep “hot” data close; batch it; move it in wider chunks.

• Concurrency: Overlap data transfers with compute (like prepping the next plate while the sauce reduces).

• Fusion: Do multiple small steps in one pass to avoid extra memory trips.

Where NVIDIA’s moat really is (beyond fast chips)

Picture a four-layer stack:

1. Hardware cadence (new chips every cycle)

2. Libraries (math + AI) that hide complexity

3. Tools (profilers, debuggers) that speed iteration

4. Ecosystem gravity: tons of examples, courses, and engineers who already know CUDA

Each layer reinforces the others. That’s why it’s hard for alternatives to catch up: they have to match all four at once.

What could chip away at the moat

• Portability efforts (like UXL/oneAPI) want one common way to program any accelerator.

• AMD’s ROCm keeps improving, especially for mainstream AI workloads.

• Code-conversion tools can move a lot of CUDA code automatically, though the last mile still needs hand work.

Practical advice: Don’t “bet the company” on a switch. Pilot one service on an alternative stack, gather real cost-at-SLO numbers, and expand only if the numbers hold.

What to watch next (if you only track three things)

1. How quickly FP4 support spreads across the libraries you use (TensorRT, cuDNN).

2. Real gains from tiles in everyday models (fewer code hacks, same or better performance).

3. Credible parity on a non-CUDA stack for your workload - at equal SLOs and total cost.

Closing thought

CUDA isn’t one trick; it’s a product system where friendlier programming (tiles), smarter chips (Blackwell), tiny numbers (FP4), and deep libraries all stack up. If you’re building today, start with the libraries, measure with Nsight, and pull the big knobs - precision, coverage, locality, fusion - before touching custom kernels. Hedge with a portability pilot if it helps you sleep at night, but make every platform decision answer the only question that matters: does this lower my cost to hit the same SLOs?

Question: Metamaterials have lived through two hype cycles already - invisibility cloaks in the 2000s, 5G antennas in the 2010s. Is the 2020s their third chance at bat - and this time, can they finally make it out of the lab and onto the balance sheet?

Decode the Basics

1. What exactly is a metamaterial, and how is it different from normal materials?

   Most materials behave the way they do because of chemistry - the atoms and molecules they’re made from. A metamaterial is different: its properties come from geometry, not chemistry.

   Imagine a chessboard patterned into a material at a scale smaller than the wavelength of light or radio waves. Those repeating structures act like “knobs” that change how waves pass through. A normal glass slab might just let light through. A metamaterial slab with engineered patterns can bend, focus, or twist that same light in ways nature never allowed.

   

   

2. How do metamaterials actually “bend” light, sound, or radio waves?
   The pop-sci gateway drug was negative-index refraction - the idea (Veselago, 1968) that if you engineer both permittivity and permeability to be negative, light can bend the “wrong” way.

   

   And how does that type of unreal refraction happen in Metamaterial? Think of a wavefront - a line of marching soldiers all stepping in sync. Now put a fence in front of them where each gate slows the soldiers down by a slightly different amount. When they regroup on the other side, the line is tilted or curved. That’s how metamaterials work: each tiny patterned cell adds a controlled phase delay. By carefully varying those delays across a surface, engineers can steer beams, focus waves, or cancel reflections. It’s the physics of delays, not bulk shape, that reshapes the wave.

3. What are the canonical demos out there: cool science vs. practical products?
   If you’ve seen the headlines about “invisibility cloaks” or “perfect lenses,” you’ve seen the hype side of metamaterials. In 2006, a Duke team demoed a rudimentary “invisibility cloak” in 2006. Below is a more recent version of it.

It worked by warping light around an object rather than reflecting it back - but only in narrow bands, with loss and noticeable distortion, so it never became a product. Still, those cloaks (and “perfect lenses”) became cultural icons that proved metamaterials can do things nature doesn’t.

The practical pivot was to metasurfaces: ultra-thin, patterned sheets that steer, focus, or filter waves with semiconductor-style manufacturing. These can replace or shrink bulky lenses and antennas and are already on commercialization paths - unlike the headline cloaks.

Where It’s Heading

4. What changed in the last 5–10 years to make metamaterials more viable?
   For years, making metamaterials was like doing art with an electron beam: slow, tiny, and expensive. That changed thanks to three big shifts:

• Deep-UV lithography (DUV): the same technology used in chip fabs now prints metasurface patterns across 8–12 inch wafers. Companies like STMicroelectronics are already running meta-optics designs on 300 mm lines - the same fabs that churn out semiconductors.

• Nanoimprint lithography (NIL): think of stamping patterns instead of drawing them. Modern NIL can press nanostructures efficiently, even with roll-to-roll approaches, opening a path to low-cost, mass production of lenses and films.

• Grayscale lithography (GSL): instead of binary on/off patterning, GSL can sculpt smooth height profiles in one shot. That allows broadband, high-efficiency lenses without stacking multiple layers.

Together, these methods are moving metasurfaces from lab demos to something fabs and foundries can scale. Add in lower-loss dielectrics and AI-assisted design, and the technology is no longer hand-crafted art.

5. What are the leading application frontiers now?
   Two stand out.

• First is RF and antennas: metamaterials enable flat, electronically steerable arrays for satellites, 5G/6G repeaters, and compact radars. Startups like Pivotal Commware, Echodyne, and Kymeta are already shipping.

  

• Second is optics: so-called “metalenses” - flat, patterned sheets that can replace multi-element glass - are now licensed into consumer and automotive supply chains through players like Metalenz.

5. What concrete inflection points are on the horizon?

• 6G standards in the late 2020s will require antennas that can steer higher-frequency signals more flexibly.

• Defense procurement is looking for compact counter-drone radars and low-SWaP (size, weight, power) sensors, where metamaterials are competitive.

• Consumer optics could break out if the first smartphone or AR device adopts meta-optics at volume.

7. What’s the realistic timeline for commercial scale?
   RF metamaterials are already commercial - you can buy a Kymeta satellite terminal or see an Echodyne radar deployed today. The next 1–3 years are about scaling in defense and telecom. Optics are ramping in the next 3–5 years as Tier-1 suppliers like ST integrate them into sensing modules for phones, cars, and AR glasses.

Zooming in to the not-there-yet product: Meta Optics

8. What can a metalens do that glass or plastic can’t?
   A traditional lens bends light because of its shape: thicker in the middle, thinner on the edges. A metalens is flat, but patterned with nanostructures that do the same bending trick - and more. It can focus, correct aberrations, and even filter polarization in one wafer-thin layer. That means fewer stacked elements, slimmer modules, and potentially cheaper assembly.

9. What are the tradeoffs?

• Pros: compactness, multifunctionality, semiconductor-style repeatability.

• Cons: lower efficiency compared to high-end glass, bandwidth limits, and fabrication challenges at visible wavelengths.

10. What does “good” look like in real products?

• In imaging sensors, a metalens needs high resolution (measured by modulation transfer function), low aberration, and thin form factors.

• In lidar and 3D sensing, beam shaping accuracy and efficiency are key.

• In phone cameras, success means shaving off millimeters of thickness without sacrificing low-light performance or adding ghosting.

Metalenz (to elaborate later) and STMicroelectronics are already pushing toward these metrics in smartphone and automotive applications. Major players in optics like Sony or Samsung also have several patents.

11. What are the common failure modes, and how do you fix them?

• Losses: use better dielectric materials to reduce absorption.

• Fabrication defects: fix with tighter process control and wafer metrology.

• Thermal drift or chromatic issues: mitigate with hybrid stacks or multi-layer designs.

Zooming out again: The market opportunities of metamaterials

12. Who are the anchor buyers?
    Defense agencies (radars, stealth), telecom and satellite operators (steerable antennas), and consumer electronics makers (smartphones, AR, automotive sensors). Each brings different price tolerance and volume potential.

13. Which incumbents are experimenting, and how does it affect startups?
    Big defense primes and telco OEMs are running pilots. On the consumer side, STMicroelectronics has licensed meta-optics IP. This is double-edged: incumbents validate the field, but once they master the process, they can pressure startup margins.

14. What near-term signals would show breakout?

• A smartphone shipping with a meta-lens.

• 6G field trials citing metasurface antennas.

• Multi-year defense production contracts for compact radars.

• Satellite terminals with metamaterial beam-steering achieving broad operator adoption.

15. Who are the startups to keep an eye on in this area ?

• Metalenz (Boston, 2016) – Probably the most visible player in meta-optics, and also the one I often encountered during my prior projects. They’ve taken Harvard lab tech into the supply chain by licensing flat “metalenses” to STMicro, aiming to slim down smartphone 3D sensing modules. Per what I heard from my friends researching in optics, they’ve been quite actively collaborating with big techs (e.g. Apple, Google) to jointly develop products that would fit into their consumer products (e.g. Smartphone, MX device.) Wonder how they are doing with those projects. Their upside to me: they have a fab-ready IP and an actual path to volume in consumer electronics. But there is still a question mark: can they can hit efficiency and bandwidth targets before handset OEMs lose patience.

  

• Imagia (Silicon Valley, 2022) – A seed-stage rival coming at optics from a different angle: building lenses like chips, patterned directly on LEDs or sensors. If it works, AR/VR hardware could suddenly shrink. The bet here is paradigm shift; the risk is classic deep-tech: long road to scale, and Metalenz already has a head start.

• Pivotal Commware (Kirkland, 2016) – A 5G infrastructure startup pushing “holographic beamforming” panels that make mmWave signals bend around corners and extend coverage. The win condition: delivering phased-array-like performance at a fraction of the cost and power. The risk: convincing conservative carriers to deploy at scale, in a market crowded with cheaper repeaters and legacy vendors.

• Echodyne (Kirkland, 2014) – Their MESA radars use metamaterials to do electronically scanned arrays without the massive price tag. That makes compact, low-power radars viable for drones, perimeter security, and defense. The upside: they already have TRL-9 deployments with defense and government customers. The risk: radar budgets are cyclical, and AESA incumbents won’t stand still.

• Radi-Cool (Boulder, 2016) – Less hyped but fascinating: a metamaterial film that cools surfaces under direct sun by radiating heat into space. Think passive air conditioning for roofs and cars. The upside: a massive climate-tech TAM. The risk: durability and adoption in the conservative building materials industry.

• Multiwave (Geneva, 2015) – Applying metamaterials to MRI. Their coils boost signal quality and enable low-field, portable MRI machines for point-of-care diagnostics. The upside: democratizing MRI in places where it’s nonexistent. The risk: brutal medical device regulatory cycles and competition from incumbent imaging players.

• PlanOpSim (Belgium, ~2018) – Less visible but strategic: they make the design software for metasurfaces, letting engineers plug flat optics into standard CAD workflows. The upside: every meta-optics team needs this bridge to real products. The risk: niche EDA markets scale only if the hardware ecosystem does.

Closing Answer

So - is this hype cycle #3, or something real?
To me, this isn’t hype-cycle #3 - it’s graduation, with guardrails. Metamaterials won’t replace glass or phased arrays across the board, but they’ll win, and stick, where thin, passive, fab-friendly parts beat incumbents on size, power, and integration. RF already crossed that line in select radars and satcom; optics follows as 300 mm meta-optics prove yield and efficiency. Watch for three tells in the next 24 months: a tier-1 handset/AR design-win, a multi-year defense production award, and 6G pilots standardizing metasurface options. Hit two of three, and we’re not in a lucky up-cycle - we’re watching niche tech harden into a moat.

Luckily consulting turned out to be a quite of a front-row seat. Over four years at McKinsey, I worked across a wide spectrum of technologies - from Asian semiconductors to an ingredient for Harry Potter’s invisible cloak - that felt one experiment away from science fiction. That kept me enthusiastic: helping technical visionaries turn lab brilliance into products and markets. I’ve helped CEOs think about partnerships, built go-to-market strategies, and modeled multi-billion-dollar autonomy roadmaps. I sometimes got feedback that I dove too deep - cornering friends to dissect a white paper diagram, or getting along a little too well with the engineering team. On nights and weekends, I built scrappy consumer apps - a spam-filter that actually worked and a wellness app that quietly found real users.

This curiosity and early exposure helped me earn the trust of skeptical engineers and hard-bar product leaders - and it still opens the door to the whiteboard. But I’ve also felt the pull to go deeper: to understand the bones and flesh of the tech and, more importantly, to nurture a product myself from day one.

This Substack is how I sharpen the muscles for that in public. I want to be closer to products that touch the real world - that means explaining complex systems clearly, making trade-offs explicit, and developing a point of view on where value will accrue.I don’t want to be the PM who only echoes what engineers say. I don’t want to be the investor who just follows hype. So I’m learning in public.

This will be part study journal, part translation service. I’ll take one hard concept at a time - from robotic stacks, invisible cloaks to innovative XR devices and services - and break it down into something we can actually understand. On top of the technical juice, I’ll add a product lens (user pain, MVP, roadmap, adoption) and a market lens (market structure, moats, where value captures).

I’m not pretending to be a deep-tech professor. I’m writing because I learn best by explaining - and because curiosity is more fun with company.

If that sounds useful, subscribe and join me. I’ll ask the confused questions so you don’t have to - and I’ll do the work to turn the answers into clear, practical takeaways.