---
title: "连续时间强化学习用于受控Hawkes跳跃扩散过程"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.19151v1"
permalink: "https://feed.astralor.com/items/itm_4db8867a1f78695d4ac68019"
published: "2026-08-19T17:40:23.000Z"
score: 73
channel: "ai"
tags: ["Hawkes-CT DDPG","Hawkes过程","强化学习","随机控制"]
---

# 连续时间强化学习用于受控Hawkes跳跃扩散过程

## AI 点评

> 这篇论文最值得关注的点在于把强化学习从离散时间的套路里拔出来，直接面向具有记忆效应的Hawkes过程。如果你是做高频交易或事件序列建模的研究者，这套无模型连续时间控制方法可能比现成的DQN类工具更贴合实际系统。

## 速览

本研究针对多变量Hawkes驱动的随机微分方程，提出在非马尔可夫环境下利用机器学习算法进行随机控制。由于Hawkes强度的路径依赖记忆性，该问题在经典随机控制理论中难以直接处理。作者首先开发了有限维马尔可夫化过程，用指数核混合逼近多变量Hawkes过程，并证明其收敛性。随后提出连续时间确定性策略梯度学习算法Hawkes-CT DDPG，仅依赖事件时间、SDE解和衰减滤波器即可实现无模型优化，无需知道Hawkes核系数。实验对比了简单指数、Erlang和幂律核下的离散时间强化学习方法，验证了算法有效性。

## 关键点

- 提出Hawkes跳跃扩散的有限维马尔可夫化方法，证明过程、强度及问题价值函数的收敛性
- 设计Hawkes-CT DDPG连续时间确定性策略梯度算法，支持无模型学习
- 算法仅需事件时间、SDE解和衰减滤波器，无需已知Hawkes核系数
- 在指数、Erlang和幂律三类核下与离散时间强化学习对比，验证优势

## 全文

We study stochastic control of multivariate Hawkes-driven stochastic differential equations with machine learning algorithms in a non-Markovian setting. Due to the path dependence of the memory of the Hawkes intensity, this problem does not fall within classical stochastic control theory outside particular Markovian kernels. We first develop a finite-dimensional Markovianization procedure and algorithm to approximate multivariate Hawkes processes with mixtures of exponential kernels. We prove the convergence of the Markovianized approximation of the Hawkes process, its intensity, and the value of the problem to the original non-Markovian processes and the value of the primal problem. We then formulate continuous-time deterministic policy gradient learning on the Markovianized approximation of the problem, called Hawkes-CT DDPG. We propose a model-free algorithm to solve the non-Markovian Hawkes-driven optimization by observing only the event times of the process, the realization of the solution to the SDE, and a chosen set of decay filters, while the Hawkes kernel coefficients remain unknown. We compare our continuous time reinforcement learning Hawkes-CT DDPG method with discrete time reinforcement learning techniques under three different types of kernels: simple exponential, Erlang, and power-law kernels.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_4db8867a1f78695d4ac68019