---
title: "VLK：从重建场景中的合成交互学习人形机器人移动操作"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2606.30645v1"
permalink: "https://feed.astralor.com/items/itm_72aa1d5b47611cd65ee5b6eb"
published: "2026-06-29T17:59:55.000Z"
score: 55
channel: "ai"
tags: ["3D高斯泼溅","Unitree","VLK","人形机器人","仿真到现实"]
---

# VLK：从重建场景中的合成交互学习人形机器人移动操作

## 速览

基于感知的人形机器人移动操作需要将自我中心观察与任务指令连接到全身运动。现有数据源无法提供完整的视觉-语言-运动学三元组。VLK通过3D高斯泼溅重建室内场景，合成导航与物体交互轨迹，生成48,000个配对轨迹，无需人工干预。训练的策略在真实Unitree G1机器人上成功执行导航与物体搬运任务，证明合成数据可有效用于仿真到现实的感知驱动机器人操作。

## 关键点

- 提出VLK框架，在重建场景中合成视觉-语言-运动学监督数据
- 使用3D高斯泼溅重建度量级室内场景，合成导航与交互轨迹
- 生成48,000个配对轨迹，无需任何人工标注
- 在真实Unitree G1机器人上验证了导航和物体搬运任务的有效性

## 全文

Perception-based humanoid loco-manipulation requires connecting egocentric observations and task instructions to whole-body motion. Learning this mapping requires synchronized egocentric images, language commands, and robot-compatible kinematic trajectories, yet no existing data source provides this complete tuple at scale. We address this bottleneck by generating vision-language-kinematics (VLK) supervision synthetically in reconstructed scenes. Our pipeline leverages 3D Gaussian Splatting to reconstruct metric-scale indoor environments, synthesizes navigation and object-interaction trajectories using privileged scene information, and renders paired egocentric observations after the fact. We produce 48,000 paired trajectories with no human intervention and train a VLK policy that predicts short-horizon whole-body kinematic trajectories. A whole-body tracker converts these predictions into actions on the physical humanoid. We evaluate on the physical Unitree G1 performing navigation and single-object transport, demonstrating that synthesized interactions in reconstructed scenes provide effective supervision for sim-to-real perception-based humanoid loco-manipulation. Project Website: https://vision-language-kinematics.github.io/

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_72aa1d5b47611cd65ee5b6eb