---
title: "学会调制而非循环：软演员-评论家算法实现逆变式热泵控制"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.09453v1"
permalink: "https://feed.astralor.com/items/itm_d97c8f81e0d61b73b3827b23"
published: "2026-08-10T11:24:29.000Z"
score: 72
channel: "tech"
tags: ["BOPTEST","PPO","Soft Actor-Critic","压缩机磨损","强化学习","热泵控制"]
---

# 学会调制而非循环：软演员-评论家算法实现逆变式热泵控制

## AI 点评

> 这篇论文提醒我们，奖励工程不能只看能耗和舒适度，还要把设备寿命这类机理成本放进去。对做建筑能源RL和热泵控制的人来说，SAC与PPO的差异是一个必须知道的坑：不是所有策略优化算法都适合处理连续动作空间里的磨损约束。

## 速览

建筑暖通空调的强化学习控制器通常只优化能耗和热舒适，却忽视了压缩机频繁启停造成的磨损。最新论文在BOPTEST热泵基准上对比SAC与PPO，发现SAC学会了持续调制压缩机功率的连续控制策略，实现每天零启停，而PPO则退化为开关控制，甚至比基线循环更频繁。在BOPTEST仿真器上，SAC策略在成本增加11.5%的情况下将热不舒适度最多降低90.7%，同时彻底消除了基线中的所有启停循环。

## 关键点

- SAC在相同MDP下学会了连续调制策略，使压缩机持续运行，实现每天零启停。
- PPO退化为开关控制，启停循环次数甚至超过基线启发式策略。
- SAC策略在成本仅增加11.5%的情况下，将热不舒适度最多降低90.7%。
- 研究基于BOPTEST bestest hydronic热泵案例，在奖励中引入压缩机磨损项。

## 全文

On--off cycling is the main cause of compressor wear in residential heat pumps, yet reinforcement learning (RL) controllers for buildings typically optimise only energy cost and thermal comfort, ignoring how much the learned policy cycles. We add a levelised compressor-wear term to the control reward and study how the resulting behaviour depends on the RL algorithm. Training Soft Actor---Critic (SAC) and Proximal Policy Optimisation (PPO) on an identical Markov decision process for the BOPTEST bestest hydronic heat pump case, we find that SAC learns a continuous modulation policy that keeps the compressor permanently engaged---the operating principle of an inverter-driven heat pump---achieving zero start-ups per day, whereas PPO collapses to bang-bang control that cycles more than the baseline. On the BOPTEST emulator the SAC policy cuts thermal discomfort by up to 90.7% for an 11.5% cost increase, while eliminating all baseline cycling.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_d97c8f81e0d61b73b3827b23