---
title: "Continuous-Time Reinforcement Learning for Controlled Hawkes Jump-Diffusions"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.19151v1"
permalink: "https://feed.astralor.com/items/itm_4db8867a1f78695d4ac68019"
published: "2026-08-19T17:40:23.000Z"
score: 73
channel: "ai"
tags: ["Hawkes-CT DDPG","Hawkes process","Reinforcement Learning","Stochastic Control"]
---

# Continuous-Time Reinforcement Learning for Controlled Hawkes Jump-Diffusions

## AI commentary

> The most noteworthy aspect is moving reinforcement learning beyond discrete-time paradigms to handle Hawkes processes with memory. If you work on high-frequency trading or event sequence modeling, this model-free continuous-time control method may be more suitable for real-world systems than off-the-shelf DQN-style tools.

## At a glance

This study addresses stochastic control of multivariate Hawkes-driven stochastic differential equations using machine learning algorithms in a non-Markovian setting. Due to the path dependence of Hawkes intensity memory, the problem does not fit classical stochastic control theory. The authors develop a finite-dimensional Markovianization procedure approximating multivariate Hawkes processes with mixtures of exponential kernels and prove convergence. They then formulate continuous-time deterministic policy gradient learning, called Hawkes-CT DDPG, which solves the non-Markovian optimization model-free by observing only event times, SDE realizations, and decay filters, without knowing Hawkes kernel coefficients. Comparisons against discrete-time reinforcement learning under exponential, Erlang, and power-law kernels demonstrate the effectiveness of the proposed method.

## Key points

- Proposes a finite-dimensional Markovianization method for Hawkes jump-diffusions, proving convergence of the process, intensity, and value function
- Designs Hawkes-CT DDPG, a continuous-time deterministic policy gradient algorithm for model-free learning
- Requires only event times, SDE solutions, and decay filters, without knowing Hawkes kernel coefficients
- Demonstrates advantages over discrete-time RL under exponential, Erlang, and power-law kernels

## Full text

We study stochastic control of multivariate Hawkes-driven stochastic differential equations with machine learning algorithms in a non-Markovian setting. Due to the path dependence of the memory of the Hawkes intensity, this problem does not fall within classical stochastic control theory outside particular Markovian kernels. We first develop a finite-dimensional Markovianization procedure and algorithm to approximate multivariate Hawkes processes with mixtures of exponential kernels. We prove the convergence of the Markovianized approximation of the Hawkes process, its intensity, and the value of the problem to the original non-Markovian processes and the value of the primal problem. We then formulate continuous-time deterministic policy gradient learning on the Markovianized approximation of the problem, called Hawkes-CT DDPG. We propose a model-free algorithm to solve the non-Markovian Hawkes-driven optimization by observing only the event times of the process, the realization of the solution to the SDE, and a chosen set of decay filters, while the Hawkes kernel coefficients remain unknown. We compare our continuous time reinforcement learning Hawkes-CT DDPG method with discrete time reinforcement learning techniques under three different types of kernels: simple exponential, Erlang, and power-law kernels.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_4db8867a1f78695d4ac68019