---
title: "Transformer引导的群智能节俭神经架构搜索"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.11826v1"
permalink: "https://feed.astralor.com/items/itm_53a10ba99007552a46a7d11e"
published: "2026-07-13T17:18:24.000Z"
score: 77
channel: "ai"
tags: ["Transformer","人工蜂群算法","神经架构搜索","群智能","边缘部署"]
---

# Transformer引导的群智能节俭神经架构搜索

## AI 点评

> 这篇论文真正打动我的是它把NAS从‘算力军备竞赛’拉回到了‘智力竞赛’。使用消费级显卡3小时就能得到一个比ResNet-20更小的模型，准确率还不差——这对小团队和边缘计算开发者简直是福音。动态熵机制的引入也很巧妙，解决了强化学习搜索中常见的早熟问题。虽然目前只在CIFAR-10和欺诈检测上测试，但思路极具扩展性，强烈推荐给所有做模型压缩或端侧AI的朋友。

## 速览

神经架构搜索（NAS）传统上需要数千GPU天的巨大算力。本研究提出一种节俭式NAS框架，将自回归Transformer控制器（通过强化学习训练）的全局宏观搜索与人工蜂群（ABC）算法的局部微观利用相结合，并在RL阶段引入动态熵机制防止早熟收敛。在消费级RTX 3060 GPU上，该方法仅用3小时就在CIFAR-10上发现仅17.4万参数、准确率84.85%的架构，显著小于ResNet-20；在信用卡欺诈检测任务上，以4600参数网络达到F1=0.71，证明了其在边缘部署中的高效性。

## 关键点

- 混合方法：Transformer的全局搜索 + ABC算法的局部微调，结合动态熵机制避免早熟收敛。
- 高效率：在消费级GPU上3小时完成搜索，CIFAR-10架构仅17.4万参数，优于基线。
- 跨领域验证：在图像分类和信用卡欺诈检测（不平衡表格数据）上均取得良好效果，支持边缘部署。
- 参数惩罚：通过算法惩罚网络深度，主动抑制模型臃肿。

## 全文

Neural Architecture Search (NAS) has automated the design of deep learning models but traditionally requires massive computational resources, often measured in thousands of GPU-days. In this paper, we propose a frugal and memetic NAS framework designed to democratize architecture design on consumer-grade hardware. Our approach combines the global macro-search capabilities of an autoregressive Transformer controller, trained via Reinforcement Learning (RL), with the local micro-exploitation of an Artificial Bee Colony (ABC) algorithm. To prevent premature convergence during the RL phase, we introduce a dynamic entropy mechanism that forces topological exploration upon detection of performance stagnation. Evaluated on a standard GPU (NVIDIA RTX 3060), our hybrid method effectively resolves the "cold-start" problem inherent in metaheuristics. By algorithmically penalizing network depth, our framework actively mitigates model bloat: on the CIFAR-10 dataset, it discovers an efficient architecture reaching 84.85% accuracy with only $\sim$174,000 parameters (significantly smaller than standard baselines like ResNet-20) in 3 hours of search time. Furthermore, we demonstrate the framework's flexibility by applying it to credit card fraud detection, directly optimizing the F1-Score on highly imbalanced tabular data to reach a F1-Score of 0.71 with a compact network of $\sim$4,600 parameters. These results suggest that our approach can yield tailored, accessible, and highly parameter-efficient deep learning models suitable for edge deployment.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_53a10ba99007552a46a7d11e