---
title: "VLK: Learning Humanoid Loco-Manipulation from Synthetic Interactions in Reconstructed Scenes"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2606.30645v1"
permalink: "https://feed.astralor.com/items/itm_72aa1d5b47611cd65ee5b6eb"
published: "2026-06-29T17:59:55.000Z"
score: 55
channel: "ai"
tags: ["3D Gaussian Splatting","Unitree","VLK","Humanoid Robots","Sim-to-Real"]
---

# VLK: Learning Humanoid Loco-Manipulation from Synthetic Interactions in Reconstructed Scenes

## At a glance

Perception-based humanoid loco-manipulation requires connecting egocentric observations and task instructions to whole-body motion. Existing data sources cannot provide the complete vision-language-kinematics tuple at scale. VLK reconstructs indoor scenes via 3D Gaussian Splatting, synthesizes navigation and object-interaction trajectories, generating 48,000 paired trajectories without human intervention. The trained policy successfully performs navigation and object transport on a physical Unitree G1 robot, demonstrating that synthetic interactions in reconstructed scenes provide effective supervision for sim-to-real perception-based humanoid loco-manipulation.

## Key points

- Proposes VLK framework to generate vision-language-kinematics supervision synthetically in reconstructed scenes.
- Uses 3D Gaussian Splatting to reconstruct metric-scale indoor environments and synthesize navigation and interaction trajectories.
- Produces 48,000 paired trajectories with no human intervention.
- Validated on a physical Unitree G1 robot for navigation and single-object transport tasks.

## Full text

Perception-based humanoid loco-manipulation requires connecting egocentric observations and task instructions to whole-body motion. Learning this mapping requires synchronized egocentric images, language commands, and robot-compatible kinematic trajectories, yet no existing data source provides this complete tuple at scale. We address this bottleneck by generating vision-language-kinematics (VLK) supervision synthetically in reconstructed scenes. Our pipeline leverages 3D Gaussian Splatting to reconstruct metric-scale indoor environments, synthesizes navigation and object-interaction trajectories using privileged scene information, and renders paired egocentric observations after the fact. We produce 48,000 paired trajectories with no human intervention and train a VLK policy that predicts short-horizon whole-body kinematic trajectories. A whole-body tracker converts these predictions into actions on the physical humanoid. We evaluate on the physical Unitree G1 performing navigation and single-object transport, demonstrating that synthesized interactions in reconstructed scenes provide effective supervision for sim-to-real perception-based humanoid loco-manipulation. Project Website: https://vision-language-kinematics.github.io/

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_72aa1d5b47611cd65ee5b6eb