---
title: "GEM-3: Timestep-Conditioned Transformers for Global Weather Forecasting"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.06241v1"
permalink: "https://feed.astralor.com/items/itm_88853788cbe5a19dd528f4eb"
published: "2026-08-06T16:27:54.000Z"
score: 66
channel: "ai"
tags: ["GEM-3","Transformer","Weather Forecasting","Timestep Conditioning","Neighborhood Attention"]
---

# GEM-3: Timestep-Conditioned Transformers for Global Weather Forecasting

## AI commentary

> This paper is insightful for weather AI, using one set of weights to flexibly switch timesteps, directly addressing the conflict between nowcasting and extended-range forecasting. It is especially important for teams developing forecasting models, and worth tracking for follow-up versions and real operational validation.

## At a glance

Existing ML weather forecasting models rely on fixed autoregressive timesteps, facing a trade-off between fine resolution and error accumulation. GEM-3 addresses this via explicit multi-timestep inference, allowing configurable timestep at run time to balance short-range usability and extended stable rollouts. Mixed-timestep training improves stability, with only ~134M parameters, delivering near-SOTA medium-range probabilistic skill and efficient training/inference.

## Key points

- GEM-3 uses a single set of weights to configure timestep at inference, from 1 to 24 hours.
- Mixed-timestep training consistently improves rollout stability over timestep-specialist models.
- Built on a lightweight neighborhood-attention transformer with ~134M parameters.
- Achieves near-SOTA medium-range probabilistic skill with stable extended rollouts and decision-relevant diagnostics.

## Full text

Existing machine-learning weather forecasting models rely on predetermined and fixed autoregressive timesteps. The choice of model timestep involves a fundamental trade-off: shorter timesteps (e.g. 1 to 6 hours) finely resolve atmospheric dynamics within the diurnal cycle but increase error accumulation for a given forecast horizon, while longer timesteps (e.g. 24 hours) reduce error accumulation but limit the usability of short-range forecasts where sub-daily predictability is high. In this work, we present GEM-3, a probabilistic global weather model that addresses this trade-off through explicit multi-timestep inference. With a single set of trained weights, the model timestep can be configured at inference time to balance predictability and usability across a broad forecast horizon. Additionally, we find that mixed-timestep training consistently improves rollout stability relative to timestep-specialist models. Under the hood, GEM-3 is a lightweight neighborhood-attention transformer with ~134M parameters on an equirectangular grid with a number of architectural advancements beyond its predecessor GEM-2. The result is a practical forecasting system that couples near-SOTA medium-range probabilistic skill, stable extended-range rollouts, efficient training and inference, and decision-relevant diagnostics.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_88853788cbe5a19dd528f4eb