Generalist Policy Deployment on Unitree G1

I extended an existing motion-imitation controller for the Unitree G1 to deploy a generalist policy — one trained to track arbitrary reference motions rather than a single hard-coded one. The hard parts weren't the model swap; they were the observation layout, the frame conventions, and the joint mapping between simulation and hardware.

Single-motion policies vs. generalist policies

The original controller runs an ONNX policy whose reference trajectory is baked into the model at training time. It's simple to deploy: feed the current state in, get joint targets out. But it's also one-trick — to switch motions, you swap policies.

A generalist policy takes the reference motion as a runtime input. The model is a TorchScript module; the motion is a .npz file containing root pose and joint positions for every frame. You can load any motion at startup and the same policy tracks it. The deployment side has to do a lot more work, though: loading and interpolating the reference, assembling a much larger observation, and keeping the proprioception history correctly buffered.

Observation layout

The generalist policy expects a 2,154-dimensional observation. It breaks down as:

[   0 :  600 ]  motion_ref   — 20 future frames × 30 dims
[ 600 :  674 ]  proprio      — angular vel, roll/pitch,
                              joint pos, joint vel, last action
[ 674 : 2154 ]  history      — last 20 proprioceptive obs

The motion reference is a sliding window of 20 future frames — the policy sees slightly into the future so it can plan stance transitions. The proprioception block is the standard locomotion observation. The history block is what makes the policy robust to noise: it sees its own recent state, not just the current instant.

Assembling this every control step requires a rolling buffer, careful frame alignment between simulator-trained data and the real IMU, and a motion library that can do SLERP interpolation between sampled frames (the motion file is at 30 fps but the controller runs at 500 Hz).

Architecture

The controller wraps the legged_rl_controllers::RlController base class and adds a parallel path for generalist mode. The two modes share the outer loop; they only diverge in how the observation gets assembled and which inference backend runs.

MotionTrackingController (extends RlController)
├── Single-motion path
│   ├── MotionOnnxPolicy       — ONNX runtime, timestep input
│   ├── MotionCommandTerm      — anchor / body tracking
│   └── ObservationManager     — standard observation assembly
│
└── Generalist path
    ├── TorchPolicy                  — LibTorch JIT inference
    ├── MotionLibrary                — .npz loading + SLERP interpolation
    ├── GeneralistMotionReferenceObs — 20 future frames (600 dims)
    ├── GeneralistProprioceptiveObs  — proprioception (74 dims)
    └── GeneralistHistoryObs         — rolling history buffer (1480 dims)

In generalist mode the controller bypasses the ObservationManager entirely — the standard pipeline assumes observation terms that don't exist for this policy. Instead the update() loop assembles the 2,154-dim vector directly, with explicit control over frame ordering and history buffering.

The control loop runs at 500 Hz. Policy inference is decimated to 50 Hz (every 10 cycles) — running the network every step is unnecessary for tracking, and the inner 500 Hz loop just resends the last computed target. This is the standard approach in real-time RL deployment and it matters a lot for jitter on the motor commands.

The four things that took the longest to find

1. The frame transform on angular velocity

getGeneralizedVelocity() returns the angular velocity of the base in the world frame. The policy was trained on IMU data, which is in the body frame. Without rotating it through the inverse base quaternion before the policy sees it, the robot fell over the instant the controller started. It looked exactly like a tuning problem and wasted a full afternoon before the frame mismatch became obvious.

ang_vel_body = base_quat.inverse() * ang_vel_world;

2. Twenty-nine joints in sim, twenty-three in the policy

The G1 simulation has 29 actuated joints. The training data excluded the six wrist joints — they aren't load-bearing and they confused the policy more than they helped. The policy outputs 23 joint targets; the controller has to expand them back into a 29-DOF command and zero (or hold) the wrists.

Wrist joints sit at indices 19–21 (left) and 26–28 (right) in the 29-DOF array. Two helpers — extract23Dofs() and expand23DofsTo29() — handle both directions, called on every observation read and every action write.

3. Ankle velocity zeroing

The training pipeline zeroed out ankle joint velocities at indices 4, 5, 10, 11 in the 23-DOF joint velocity vector. This was an artifact of how the simulator modeled ankle compliance. The policy learned to expect zeros there. If you feed it the real ankle velocities at deploy time, the gait becomes visibly jittery — not catastrophic, but not the smooth tracking the sim showed either.

4. Action processing

The policy outputs raw actions. The deployed pipeline is:

action  →  clip([-10, 10])  →  × 0.5  →  + default_joint_pos

Clipping prevents pathological outputs from any one inference step. The 0.5 scale matches the action scale used during training. Adding back the default pose gives the policy a useful prior — it learns deltas from a known-stable pose, not absolute joint angles.

PD gains

These are set once at startup and held for the full episode. They started from the main-branch values and were reduced 50–70% — the generalist policy is more confident about its targets than the single-motion policy, so the controller doesn't need to fight the policy as hard.

What worked

The robot tracks runtime-loaded motions cleanly. Switching motions is now a matter of pointing the launch file at a different .npz instead of retraining a policy. The 50 Hz inference / 500 Hz motor command split keeps the motors smooth without burning compute on redundant network passes.

The single-motion path is untouched — both modes coexist in the same controller binary, branching on a single use_generalist launch parameter. This was important: the lab still uses the ONNX path for several existing experiments, and breaking it would have been costly.

Code

The full controller is open source:

github.com / Abhimanyu-0 / motion_tracking_controller↗

The README in that branch covers build, launch, and motion-file format in detail.