---
title: "Transformer-Guided Swarm Intelligence for Frugal Neural Architecture Search"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.11826v1"
permalink: "https://feed.astralor.com/items/itm_53a10ba99007552a46a7d11e"
published: "2026-07-13T17:18:24.000Z"
score: 77
channel: "ai"
tags: ["Transformer","Artificial Bee Colony","Neural Architecture Search","Swarm Intelligence","Edge Deployment"]
---

# Transformer-Guided Swarm Intelligence for Frugal Neural Architecture Search

## AI commentary

> What truly impresses me about this paper is that it brings NAS back from a 'compute arms race' to an 'intelligence contest'. With a consumer-grade GPU, you can get a model smaller than ResNet-20 in 3 hours with competitive accuracy—a game-changer for small teams and edge computing developers. The dynamic entropy mechanism cleverly addresses premature convergence in RL-based search. Though only tested on CIFAR-10 and fraud detection, the methodology is highly extensible; highly recommended for anyone working on model compression or on-device AI.

## At a glance

Neural Architecture Search (NAS) traditionally requires massive computational resources, often measured in thousands of GPU-days. This paper proposes a frugal NAS framework combining the global macro-search of an autoregressive Transformer controller (trained via RL) with the local micro-exploitation of an Artificial Bee Colony (ABC) algorithm. A dynamic entropy mechanism prevents premature convergence during RL. On a consumer-grade RTX 3060 GPU, it discovers an efficient architecture on CIFAR-10 with only ~174K parameters and 84.85% accuracy in 3 hours, significantly smaller than ResNet-20. For credit card fraud detection, it achieves an F1-score of 0.71 with a compact network of ~4,600 parameters, demonstrating suitability for edge deployment.

## Key points

- Hybrid method: global macro-search via Transformer + local micro-exploitation via ABC, with dynamic entropy to avoid premature convergence.
- High efficiency: 3-hour search on consumer GPU yields CIFAR-10 architecture with 174K params, outperforming baselines.
- Cross-domain validation: good results on image classification and credit card fraud detection (imbalanced tabular data), enabling edge deployment.
- Depth penalty: algorithmically penalizes network depth to mitigate model bloat.

## Full text

Neural Architecture Search (NAS) has automated the design of deep learning models but traditionally requires massive computational resources, often measured in thousands of GPU-days. In this paper, we propose a frugal and memetic NAS framework designed to democratize architecture design on consumer-grade hardware. Our approach combines the global macro-search capabilities of an autoregressive Transformer controller, trained via Reinforcement Learning (RL), with the local micro-exploitation of an Artificial Bee Colony (ABC) algorithm. To prevent premature convergence during the RL phase, we introduce a dynamic entropy mechanism that forces topological exploration upon detection of performance stagnation. Evaluated on a standard GPU (NVIDIA RTX 3060), our hybrid method effectively resolves the "cold-start" problem inherent in metaheuristics. By algorithmically penalizing network depth, our framework actively mitigates model bloat: on the CIFAR-10 dataset, it discovers an efficient architecture reaching 84.85% accuracy with only $\sim$174,000 parameters (significantly smaller than standard baselines like ResNet-20) in 3 hours of search time. Furthermore, we demonstrate the framework's flexibility by applying it to credit card fraud detection, directly optimizing the F1-Score on highly imbalanced tabular data to reach a F1-Score of 0.71 with a compact network of $\sim$4,600 parameters. These results suggest that our approach can yield tailored, accessible, and highly parameter-efficient deep learning models suitable for edge deployment.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_53a10ba99007552a46a7d11e