---
title: "GPU Parallelization Strategies for Forward and Backward Propagation in Shallow Neural Networks: A CUDA-Based Comparative Study"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2606.30497v1"
permalink: "https://feed.astralor.com/items/itm_d18e08d7e1f9eae33c7854c6"
published: "2026-06-29T16:02:10.000Z"
score: 53
channel: "tech"
tags: ["CUDA","GPU Parallelization","NVIDIA","Memory Access Optimization","Shallow Neural Network"]
---

# GPU Parallelization Strategies for Forward and Backward Propagation in Shallow Neural Networks: A CUDA-Based Comparative Study

## At a glance

This paper presents a comparative study of three CUDA optimization strategies for forward and backward propagation in shallow neural networks: tiled shared memory with +1-column padding to eliminate bank conflicts, pre-transposed weight matrices for coalesced global memory access, and a fused MatMul+ReLU kernel to reduce intermediate global memory round-trips. Experiments on an NVIDIA Tesla T4 show that the fully optimized implementation achieves a 1.41x speedup over the baseline CUDA version on the large dataset (25,600 samples), reducing execution time from 21.0s to 14.8s. Results are compared against a sequential CPU baseline and an OpenMP parallel implementation, demonstrating the effectiveness of memory-access optimization in GPU-accelerated deep learning primitives.

## Key points

- Proposes three CUDA optimization strategies: tiled shared memory, pre-transposed weight matrices, and fused MatMul+ReLU kernel.
- Achieves 1.41x speedup on large dataset, reducing execution time from 21.0s to 14.8s.
- Compared with CPU and OpenMP baselines, validates the critical role of memory-access optimization in GPU acceleration.

## Full text

We present a comparative study of CUDA optimization strategies applied to forward and backward propagation in a shallow neural network. Three stacked optimizations are evaluated: (1) tiled shared memory with bank-conflict elimination via +1-column padding, (2) pre-transposed weight matrices for coalesced global memory access, and (3) a fused MatMul+ReLU kernel that eliminates intermediate global-memory round-trips. Experiments on an NVIDIA Tesla T4 (CUDA 13.0) across three dataset sizes show that the fully optimized implementation achieves a 1.41x speedup over the baseline CUDA version on the large dataset (25,600 samples), reducing execution time from 21.0s to 14.8s. Results are compared against a sequential CPU baseline and an OpenMP parallel implementation, demonstrating the effectiveness of memory-access optimization in GPU-accelerated deep learning primitives.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_d18e08d7e1f9eae33c7854c6