---
title: "Safe Multi-Objective Reinforcement Learning: Zero Constraint Violation and Primal-Averaging Convergence"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2610.39409v1"
permalink: "https://feed.astralor.com/items/itm_07991b41e7795031473b4314"
published: "2026-10-29T05:47:57.000Z"
score: 56
channel: "tech"
tags: ["安全","RAG","Safe Multi-Objective Reinforcement","Learning","Zero Constraint Violation","Primal-Averaging Convergence Many","MORL","Many"]
---

# Safe Multi-Objective Reinforcement Learning: Zero Constraint Violation and Primal-Averaging Convergence

## 速览

Many applications of multi-objective reinforcement learning (MORL) require policies that balance conflicting objectives while satisfying safety constraints. We study safe MORL with zero constraint violation throughout l…

## 关键点

- Many applications of multi-objective reinforcement learning (MORL) require policies that balance conflicting objectives while satisfying safety constraints.
- We study safe MORL with zero constraint violation throughout learning, covering both cumulative and stepwise constraints.
- A key challenge is to explore sufficiently under unknown dynamics and constraint costs without violating the constraints.

## 全文

Many applications of multi-objective reinforcement learning (MORL) require policies that balance conflicting objectives while satisfying safety constraints. We study safe MORL with zero constraint violation throughout learning, covering both cumulative and stepwise constraints. A key challenge is to explore sufficiently under unknown dynamics and constraint costs without violating the constraints. For a given preference over objectives, we develop an algorithm to learn a safe Pareto optimal policy by combining optimistic reward estimates with pessimistic cost bounds. With high probability, every policy executed over $T$ episodes incurs zero constraint violation. The algorithm returns a safe Markov policy induced by the averaged occupancy measure and achieves an $\widetilde{O}(1/\sqrt{T})$ suboptimality gap under smooth Tchebycheff scalarization. We further develop a preference-free algorithm that first explores safely without a prespecified preference and then computes a safe policy for any given preference without further interaction. Over $T$ exploration episodes and $K$ planning iterations, it retains the same safety guarantee and achieves an $\widetilde{O}(1/\sqrt{T}+1/\sqrt{K})$ suboptimality gap. Both algorithms require knowledge of a policy that strictly satisfies the constraints and its costs. We then relax these requirements, incurring at most constant cumulative violation. To our knowledge, we provide the first sample-efficient MORL algorithms that guarantee safety for every executed policy.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_07991b41e7795031473b4314