---
title: "浅层神经网络前向与反向传播的GPU并行化策略：基于CUDA的对比研究"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2606.30497v1"
permalink: "https://feed.astralor.com/items/itm_d18e08d7e1f9eae33c7854c6"
published: "2026-06-29T16:02:10.000Z"
score: 53
channel: "tech"
tags: ["CUDA","GPU并行化","NVIDIA","内存访问优化","浅层神经网络"]
---

# 浅层神经网络前向与反向传播的GPU并行化策略：基于CUDA的对比研究

## 速览

本论文针对浅层神经网络中的前向和反向传播，系统评估了三种CUDA优化策略：平铺共享内存结合+1列填充消除bank冲突、预转置权重矩阵实现合并全局内存访问、融合MatMul+ReLU内核减少中间全局内存往返。在NVIDIA Tesla T4上的实验表明，全优化版本在大数据集（25,600样本）上比基线CUDA版本获得1.41倍加速，执行时间从21.0秒降至14.8秒。结果与顺序CPU和OpenMP并行实现对比，凸显了GPU加速深度学习原语中内存访问优化的重要性。

## 关键点

- 提出三种CUDA优化策略：平铺共享内存、预转置权重矩阵、融合MatMul+ReLU内核
- 在大数据集上实现1.41倍加速，执行时间从21.0秒降至14.8秒
- 与CPU和OpenMP基线对比，验证内存访问优化在GPU加速中的关键作用

## 全文

We present a comparative study of CUDA optimization strategies applied to forward and backward propagation in a shallow neural network. Three stacked optimizations are evaluated: (1) tiled shared memory with bank-conflict elimination via +1-column padding, (2) pre-transposed weight matrices for coalesced global memory access, and (3) a fused MatMul+ReLU kernel that eliminates intermediate global-memory round-trips. Experiments on an NVIDIA Tesla T4 (CUDA 13.0) across three dataset sizes show that the fully optimized implementation achieves a 1.41x speedup over the baseline CUDA version on the large dataset (25,600 samples), reducing execution time from 21.0s to 14.8s. Results are compared against a sequential CPU baseline and an OpenMP parallel implementation, demonstrating the effectiveness of memory-access optimization in GPU-accelerated deep learning primitives.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_d18e08d7e1f9eae33c7854c6