---
title: "Daedalus-150M: A Convolution-Attention Hybrid Designed for CPU Inference"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.20210v1"
permalink: "https://feed.astralor.com/items/itm_2040d0cef604c3a2e9e56b8c"
published: "2026-08-20T16:09:43.000Z"
score: 77
channel: "ai"
tags: ["CPU Inference","Daedalus-150M","Convolution-Attention Hybrid","Small Language Model"]
---

# Daedalus-150M: A Convolution-Attention Hybrid Designed for CPU Inference

## AI commentary

> What stands out here is not benchmark topping but using deployment constraints as the design starting point, not an afterthought. For teams working on edge AI, embedded systems, or privacy-preserving computing, the fact that the speed advantage grows with context is particularly valuable—longer conversations translate into bigger relative gains. If you have a CPU inference workload, it's worth reproducing its bandwidth analysis on your own hardware.

## At a glance

Daedalus-150M is a convolution-attention hybrid small language model designed explicitly for CPU inference. Instead of building a large model and compressing it later, the authors fixed the target first: one user, one token at a time, 4-bit weights, and an ordinary CPU. The architecture uses full attention in only 6 of its 18 blocks; the other 12 rely on short convolutions whose cache stays two timesteps wide regardless of conversation length. Trained from scratch on 59.9B tokens, it scores 47.31 on a five-task benchmark against a 42.20 baseline, beating GPT-2 124M, Pythia-160M, OPT-125M, and GPT-neo-125M, and exceeding MobileLLM-125M's published score. Compared with a conventional all-attention model of the same size on the same data, the hybrid gains 0.81% quality, produces a 6.3% smaller 4-bit file, and decodes 1.76x faster at 2048 tokens of context. The speed edge grows with sequence length, which a simple bandwidth calculation cannot fully explain. The paper also candidly reports failures: unmitigated 4-bit quality loss, about half the convolution channels ending up inert, and an oversized vocabulary.

## Key points

- Architecturally, only 6 of 18 blocks use full attention; the other 12 use short convolutions, keeping cache size constant regardless of context length.
- After training on 59.9B tokens, it scores 47.31 on a five-task benchmark, surpassing several larger models trained on more data.
- Compared with an equal-sized all-attention model, it produces a 6.3% smaller 4-bit file and decodes 1.76x faster at 2048-token context, with the edge growing as context lengthens.
- The paper candidly reports failures: unmitigated 4-bit quality loss, roughly half the convolution channels ending up inert, and an oversized vocabulary.

## Full text

Small language models are usually built like large ones and then squeezed onto a CPU afterwards. We did the opposite: we fixed the target first, one user, one token at a time, 4-bit weights, ordinary CPU, and chose the architecture to suit it. The result keeps full attention in only 6 of its 18 blocks. The other 12 use short convolutions whose memory is two timesteps wide no matter how long the conversation gets, so two thirds of the network never re-reads a growing cache. Trained from scratch on 59.9B tokens, the model scores 47.31 on a five-task benchmark against a bar of 42.20 that was fixed before training began. It beats GPT-2 124M, Pythia-160M, OPT-125M and GPT-neo-125M, all trained on three to six times more data, and exceeds MobileLLM-125M's published score despite that model seeing a trillion tokens. Validation bits-per-byte is 0.8685. To check the architecture rather than the training recipe, we trained a conventional all-attention model of the same size on the same data, and wrote down the winning condition before scoring either. The hybrid won the chosen quality metric by 0.81%, matched it on downstream tasks, produced a 6.3% smaller 4-bit file, and decoded 1.76x faster at 2048 tokens of context, 2.08x against an external model of similar size. In every measurement the speed advantage is near zero at an empty context and grows with length, which is what the mechanism predicts and what a merely leaner model would not show. A simple bandwidth calculation predicts only 1.17x, so memory volume alone does not explain the gap. We also report what did not work: an unmitigated 4-bit quality cost, roughly half the convolution channels ending up inert and impossible to remove, and a vocabulary larger than this model size warrants.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_2040d0cef604c3a2e9e56b8c