---
title: "Behavioral Steering of Vision-Language-Action Models in the Low-Parameter Regime"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2611.00522v1"
permalink: "https://feed.astralor.com/items/itm_91529c21eb5264efc05924f0"
published: "2026-10-31T01:23:01.000Z"
score: 56
channel: "ai"
tags: ["多模态","Behavioral Steering","Vision-Language-Action Models","Low-Parameter Regime Reinforcement","VLAs","VLA","Reinforcement"]
---

# Behavioral Steering of Vision-Language-Action Models in the Low-Parameter Regime

## 速览

Reinforcement learning (RL) is increasingly used to adapt pretrained vision-language-action models (VLAs). We investigate the problem of behavioral steering: adapting a VLA's execution style to preferences such as carry…

## 关键点

- Reinforcement learning (RL) is increasingly used to adapt pretrained vision-language-action models (VLAs).
- We investigate the problem of behavioral steering: adapting a VLA's execution style to preferences such as carrying objects higher.
- To investigate how little trainable capacity such steering requires and how much pretrained capability can be retained, we freeze an OpenVLA-OFT 7B policy and optimize TinyLoRA adapters with 8 to 384 trainable parameter…

## 全文

Reinforcement learning (RL) is increasingly used to adapt pretrained vision-language-action models (VLAs). We investigate the problem of behavioral steering: adapting a VLA's execution style to preferences such as carrying objects higher. To investigate how little trainable capacity such steering requires and how much pretrained capability can be retained, we freeze an OpenVLA-OFT 7B policy and optimize TinyLoRA adapters with 8 to 384 trainable parameters using either policy-gradient RL (GRPO) or evolutionary search (CMA-ES). With just 256 trainable parameters, we raise carry height by 120 mm on trained tasks at 92% success and by 74 mm on a held-out task at 80% success. By rescaling the learned update, we can tune the trade-off between style gains and capability retention without retraining. Increasing adapter capacity generally improves this trade-off, retaining more pretrained capability at comparable style gains. The choice of optimizer also depends on capacity, with CMA-ES being competitive at low parameter counts, while GRPO offers better trade-offs as capacity grows. These findings show that extremely low-dimensional updates can steer pretrained VLA behavior, with adapter capacity, optimizer choice, and update scaling offering practical control over adaptation and retention.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_91529c21eb5264efc05924f0