What you get today

The first version of Bellwether ends with a plan. You describe a process, and Bellwether builds the workflow. It names the person, AI software or robot for each task, lists the parts each one needs, and simulates every robot step in NVIDIA Isaac Sim with physics-based objects and environments. You can open any task to watch its simulation, or watch a lifelike video of the whole workflow running.

That plan is useful on its own. You can take it to leadership to get a project funded. You can give it to an integrator, who can then quote the build instead of discovering it, or send it to a robot or tooling maker so they know exactly what you need. Many automation projects stall before this point because nobody can say clearly what the robot should do, with which parts, and whether it is possible at all.

Why a plan isn’t a robot yet

A simulation is built to show the process. A robot needs a control policy that holds up on real hardware. Policies trained in simulation often fail when they meet a real building, even though nothing in the code is broken. The simulated world differs from the real one in friction, lighting, sensor noise, timing and many other small ways. This is the sim-to-real gap, and closing it is the core science 3GEN works on.

The gap comes from several distinct places, and each one needs a different fix. We explain where it originates and the main methods that close it on our Sim-to-Real Engine page. The rest of this page covers how Bellwether applies that work.

Grading the simulation

Before anyone spends compute training on a simulation, Bellwether checks whether it would work at all. It grades every simulation against its target, the job the task actually has to do. A weak grade comes with suggestions, such as more detail about the object, a better scan, or a more specific description of the environment.

We call the distance between a simulation and its target the sim-to-target gap. Measuring it well is new science. You have to define what a perfect run of a task looks like, find where a simulation departs from it, and correct the difference. A grade also tells you early when a task sits too far from its target to be worth training on, before you spend weeks of compute on it.

Turning simulations into training data

A robot learns a task from examples, and collecting examples on real hardware is slow. It usually means weeks of a person guiding the robot in the target cell before the first policy exists. Simulation can produce examples far faster. The next version of Bellwether will take a graded simulation and run it thousands of times, changing the lighting, object positions, weights, friction and other conditions on each run, to generate synthetic training data for the robot’s model.

The technique is called domain randomization, and it has a known trap. If the ranges are too broad, the robot learns to hedge against conditions that never occur and gets worse at the real job. Grading each simulation against its target first keeps the randomization grounded: we vary what actually varies on your floor, by about as much as it varies.

The model on the robot

The resulting model runs in two tiers. A small model on the robot’s own computer handles the routine case and reacts in milliseconds. When it meets something it doesn’t recognize, it hands the decision to a large model running in the cloud or on site.

Our research on World Action Models belongs in that second tier. A world action model predicts how the scene will change and what the robot should do about it in a single model, which gives it the physical reasoning edge cases need. It is also too slow to run on every routine cycle, so the split puts it where its speed is acceptable. We cover the tradeoff in World action models and the economics of robot deployment.

Once the robot is working, every real run feeds back into training, failures included. Each time the small model hands a situation to the large one, that hand-off records where the gap still exists. The gap shrinks the longer the robot runs.

What to do with your plan now

A complete plan with simulations is worth having before the later steps ship. An integrator can start from your workflow, parts list and simulations instead of a discovery phase. A robot maker can tell you right away whether their hardware fits the step. Where a part doesn’t exist yet, 3GEN’s professional services can help design around the gap.

The workflow and simulations you build now are also the starting point for training data and a deployed model as those steps become available.