Project EREN: My Journey to Building a CUDA-Powered LLM Inference Engine

Описание и применение

Создана базовая инфраструктура для разработки инференс‑движка LLM на CUDA: структура проекта, CMake, простые ядра (сложение векторов, редукция), обработка ошибок и тесты.

Задача

Understand and implement the components that run large language models on GPU, rather than relying on existing inference frameworks.

Результат

Established a reliable C++/CUDA development foundation with basic kernels, error handling, and testing, enabling progression to a GPT‑2 reference implementation.

Рабочий процесс

• Set up repository structure and CMake build system
• Implemented basic CUDA vector addition kernel to validate memory allocation and kernel launch
• Implemented shared‑memory multi‑stage reduction kernel
• Created reusable CUDA error‑checking helper
• Added CTest smoke tests and initial reduction benchmark
• Verified clean build and passing tests via terminal output

Доказательства

My first milestone was establishing a reliable C++/CUDA development foundation.

Источник

Implemented a basic vector addition kernel to validate device memory allocation, host-to-device transfers, kernel launches, and result verification.

Источник

Implemented a shared-memory, multi-stage reduction baseline.

Источник

Created a reusable CUDA error-checking helper to make runtime failures easier to identify and debug.

Источник

Added smoke tests using CTest and an initial reduction benchmark target.

Источник