Vision-to-Action Policy on Unitree G1

A vision-only student policy for the Unitree G1, distilled from a privileged motion-imitation teacher (TWIST). The student takes proprioception plus a 512-dim feature extracted from a single video frame and outputs 23 joint targets at 50 Hz. The teacher needs ground-truth future motion to track. The student does not — which is the whole point.

The problem

TWIST tracks reference motions beautifully in simulation. To do so it reads future motion targets at every timestep — root pose, joint positions, body keypoints, all looked up from the reference trajectory. That works in sim, where you have the trajectory pre-loaded. On a real robot, you don't have a ground-truth future to look up.

The question becomes: can the privileged motion references be replaced with something a real robot can actually observe? If yes, the policy becomes deployable.

The idea — vision as a substitute for future references

PhysHMR proposes that monocular video of the demonstrated motion encodes essentially the same intent the teacher reads off the privileged references. Pass each frame through a pre-trained video-based motion recovery network (GVHMR) and get a 512-dim feature vector that compactly represents the human's pose and short-term motion. Use that as the student's "what should I be doing" signal in place of the future-reference block.

If this works, the deployment story becomes simple: a camera watches the operator perform the motion, GVHMR turns frames into features in real time, the student fuses them with its own IMU and joint encoders and decides what to do next. No motion library on the robot.

Pipeline at a glance

Human3.6M video  ──►  GVHMR encoder (offline)  ──►  per-frame 512-dim features
                                                            │
H3.6M mocap   ────────────────────────────────────►  motion .pkl
                                                     (kinematics + img_features)
                                                            │
                                                            ▼
              MotionLib  (loads .pkl, interpolates GVHMR features over time)
                                       │
                                       ▼
                G1MimicDistill env  (obs_type = physhmr_student)
                                       │
                                       ▼   obs = [ proprio(74) | img_features(512) ]   →  586
                                       │
                ActorCriticVision  (MLP, no motion encoder)
                                       │
                                       ▼   23 joint position targets
                                       │
                DAgger PPO  (KL to TWIST teacher, BC coef annealed)
                                       │
                                       ▼
                          TorchScript export → MuJoCo / real-robot deploy

Observation space — 586 dims

The first 74 dims are standard locomotion proprioception. The last 512 are GVHMR features from a single future frame.

A few small but load-bearing details:

• The vision frame is t+1, not t. History windows of frames were tried and produced "walk in place" behaviour. A single look-ahead frame biases the policy toward forward motion — it sees what the human will be doing one step into the future, not what they're doing now.
• Ankle velocities are zeroed at indices 4, 5, 10, 11. Too noisy to use raw; same convention as the teacher was trained with.
• No 6D rotation, no history buffer. IMU roll/pitch and the GVHMR features carry enough orientation and temporal information on their own.
• Proprio noise ramps over training. The policy first learns the task with clean state, then hardens to sensor noise as training proceeds.

What the student does not see

The whole point of distillation is restricting the student to things a real robot can measure. The student has no access to:

• Future motion references — the 1,160-dim mimic observation the teacher uses
• Ground-truth root velocity or height
• Body keypoint positions
• Privileged simulator state (mass, friction, motor strength)
• Any explicit history buffer

Architecture

Actor   :  MLP   586 ─► 512 ─► 512 ─► 256 ─► 128 ─► 23
Critic  :  MLP  1318 ─► 512 ─► 512 ─► 256 ─► 128 ─►  1     (privileged, training-only)

Activation     :  SiLU
LayerNorm      :  on
Action std     :  per-joint, learnable
                  (init: 0.7 legs, 0.4 torso, 0.5 arms)

A plain MLP — no motion encoder, no recurrence. The GVHMR features themselves already carry temporal context (GVHMR is trained on video clips, not single images), so the policy doesn't need to maintain its own state.

The actor is the deployable model. The critic, used only during training, gets the full 1,318-dim privileged observation the teacher had access to — a standard asymmetric actor-critic setup. The critic's value estimate stays well-grounded; the actor learns to produce the same actions from much less information.

Training — DAgger PPO

The loss is PPO plus a KL term against the teacher:

L  =  L_PPO  +  λ_dagger(t) · KL( π_student(·|o_student) ‖ π_teacher(·|o_teacher_priv) )

Both policies are queried in lock-step on every rollout step. The teacher reads its full privileged observation; the student reads the 586-dim deployable observation; we penalize divergence between their action distributions. As training progresses the BC weight anneals, handing control back to PPO and the environment reward.

The first 100 iterations roll out the teacher alone to seed on-policy data. After that the student takes over, with the KL term keeping its action distribution close to the teacher's during the early phase when its policy is still mostly noise.

Reward retuning

The student inherits the teacher's motion-tracking reward, but two terms had to be retuned hard because of what the student can no longer see:

Root velocity weight: 1.0 → 15.0

Without future root-position references, the policy will happily satisfy most of the tracking reward by walking in place. Aggressively rewarding forward velocity forces the policy to actually translate.

Global position rewards: disabled

Position-matching rewards (global_root_xy, global_feet_xy) were tried and turned off. The reference motion lags the simulated robot by a few frames; position matching therefore pulls the robot backward, toward where the reference thinks it should still be. Velocity matching is lag-robust and does the right thing.

Dataset — Human3.6M + GVHMR

611 motion sequences from H3.6M, S01–S11 actors across roughly nine actions each and multiple camera views. Each motion .pkl carries the standard arrays (root pose, joint angles, frame rate) plus an img_features array of shape (num_frames, 512) — GVHMR features extracted offline from the matching camera-view video.

The motion library was extended to load and interpolate these features on the fly. The motion runs at H3.6M's native frame rate; the controller runs at 50 Hz. At every query, the library linearly interpolates between the two adjacent feature vectors so the vision input stays temporally aligned with the kinematic motion at non-integer timestamps.

Deployment

TorchScript export

After training, the actor exports to TorchScript — actor only, no critic, no motion encoder. The export script auto-detects the policy class and infers observation / action dimensions directly from the loaded checkpoint, so the same script handles both the old motion-encoder student and the new vision-only one.

MuJoCo verification

A separate MuJoCo harness loads the exported policy and runs it against a stripped-down motion library that does not depend on IsaacGym (working around the well-known "PyTorch imported before IsaacGym" import-order failure). The observation builder mirrors the training-time one exactly:

obs[  0:  3]  =  base_ang_vel * 0.25
obs[  3:  5]  =  roll, pitch                    (from base_quat → euler)
obs[  5: 28]  = (dof_pos − default_dof_pos)
obs[ 28: 51]  =  dof_vel * 0.05                 (ankle joints zeroed)
obs[ 51: 74]  =  last_action
obs[ 74:586]  =  motion_lib.get_image_features(motion_id, motion_time)

Real robot

Three things have to come together on hardware:

1. GVHMR running in real time at ≥50 Hz on an operator-view camera, emitting 512-dim features per frame. Currently the bottleneck.
2. IMU + encoder packing matching the training-time layout. The existing teacher-deployment machinery already does this and can be reused as-is.
3. A 50 Hz control loop fusing the two streams and shipping joint targets to the motors.

Design decisions worth calling out

1. One future frame, not a history window. Vision history under-translates; a single look-ahead frame encodes forward intent and keeps the observation small.
2. No 6D rotation observation. An early plan included one. IMU roll/pitch turned out to be sufficient. The 6D rotation helper still lives in the codebase but is unused.
3. Asymmetric actor-critic. Critic keeps the full privileged observation throughout training; only the actor is constrained to deployable inputs. Standard for sim-to-real, and it keeps value estimates well-grounded under the impoverished student input.
4. Velocity matching, no global position. The dominant reward signal for forward motion. Global position rewards are actively harmful given the reference-vs-sim timing lag.
5. Pure MLP, no recurrence. GVHMR features already encode a clip, so the policy doesn't need to maintain temporal state of its own.

Status

Built:

• Environment, observation builder, training config
• Motion library extension that loads and interpolates GVHMR features
• ActorCriticVision policy class + DAgger training pipeline
• TorchScript export script with policy-class auto-detection
• MuJoCo verification harness, IsaacGym-free

Open:

• Full training run + checkpoint to commit to
• Quantitative eval vs. the teacher — tracking error, survival, drift
• Live GVHMR pipeline on the real robot
• Domain-randomization ablation for the vision-only policy

References

• PhysHMR↗ — the design inspiration
• GVHMR — the vision encoder
• TWIST — the privileged teacher policy
• Ross et al., DAgger
• Human3.6M — the source dataset

Code

Code release coming once the training run is finalized and the real-robot pipeline is end-to-end.