---
title: "Towards an Optimal Control Foundation of Early Exits - Turnpikes and ResNets"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2611.05120v1"
permalink: "https://feed.astralor.com/items/itm_ab2e18ea143285c73e1a0952"
published: "2026-11-03T16:08:15.000Z"
score: 58
channel: "tech"
tags: ["推理效率","Towards","Optimal Control Foundation","Early Exits","Turnpikes","ResNets The","ResNets"]
---

# Towards an Optimal Control Foundation of Early Exits - Turnpikes and ResNets

## 速览

The feed-forward propagation through neural networks is crucial for training and inference. To trade off efficiency and accuracy, it has been proposed to consider intermediate outputs of hidden states as data-dependent …

## 关键点

- The feed-forward propagation through neural networks is crucial for training and inference.
- To trade off efficiency and accuracy, it has been proposed to consider intermediate outputs of hidden states as data-dependent regularization during training and, ultimately, as early-exits during inference.
- In this work, we analyze the joint optimization of loss functions computed for the output layer and intermediate layers using optimal control theory.

## 全文

The feed-forward propagation through neural networks is crucial for training and inference. To trade off efficiency and accuracy, it has been proposed to consider intermediate outputs of hidden states as data-dependent regularization during training and, ultimately, as early-exits during inference. In this work, we analyze the joint optimization of loss functions computed for the output layer and intermediate layers using optimal control theory. We prove that ResNets trained in this joint fashion exhibit the so-called turnpike phenomenon, this implies that the number of layers required to reach a prescribed loss level is bounded from above independently of the overall network depth. This induces a similarity property across the internal representations of early-exiting networks of varying depths. We connect the turnpike to the overthinking phenomenon, and provide theoretical insights into the success of early-exit criteria derived from softmax probabilities, specifically entropy and max-probability. In a range of experiments, we demonstrate the existence and effect of turnpikes and the resulting similarity in the early-exit performance for ResNets of varying depths.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_ab2e18ea143285c73e1a0952