---
title: "DES-JEPA: Learning a Shared Vocabulary for High-Dimensional Discrete Event Sequences"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2610.38445v1"
permalink: "https://feed.astralor.com/items/itm_0fa4dd948dd96eb12aa315b0"
published: "2026-10-28T08:24:09.000Z"
score: 54
channel: "tech"
tags: ["推理效率","DES-JEPA","Learning","Shared Vocabulary","High-Dimensional Discrete Event","Sequences Temporal","Chronos","TimesFM"]
---

# DES-JEPA: Learning a Shared Vocabulary for High-Dimensional Discrete Event Sequences

## At a glance

Temporal foundation models such as Chronos and TimesFM generalize across domains through a shared numerical observation space. In contrast, discrete event sequences (DESs), including operational event logs or patient cl…

## Key points

- Temporal foundation models such as Chronos and TimesFM generalize across domains through a shared numerical observation space.
- In contrast, discrete event sequences (DESs), including operational event logs or patient clinical-event trajectories, have domain-specific event vocabularies whose labels lack a consistent correspondence across domains.
- Beyond this lack of consistency, the number of distinct events in real-world settings is high enough to challenge both classical temporal point process modeling and current in-context foundation inference models such as…

## Full text

Temporal foundation models such as Chronos and TimesFM generalize across domains through a shared numerical observation space. In contrast, discrete event sequences (DESs), including operational event logs or patient clinical-event trajectories, have domain-specific event vocabularies whose labels lack a consistent correspondence across domains. Beyond this lack of consistency, the number of distinct events in real-world settings is high enough to challenge both classical temporal point process modeling and current in-context foundation inference models such as FIM-PP. Critically, current foundation models for DESs are limited to a small, fixed number of event types, restricting their applicability to realistic, high-dimensional event streams. To address this, we propose two complementary modules: (i) DES-JEPA, a joint-embedding predictive architecture that learns a shared discrete vocabulary from synthetic stochastic processes with unaligned event vocabularies; and (ii) DES-PFN, a discrete event sequence prior-fitted network that reuses the learned vocabulary for in-context next-event prediction. Without task-specific fine-tuning, this framework achieves competitive zero-shot performance on eight unseen real-world datasets and scales up to 700 distinct event types.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_0fa4dd948dd96eb12aa315b0