My Journey Into Learning About ML and Implementing a MLP From Scratch

Description and use

Implemented a Julia MLP achieving 99.46% MNIST accuracy in ~2 min using custom code, data augmentation, and momentum.

Problem

Achieve high accuracy on MNIST with fast training using minimal libraries and a new language (Julia).

Outcome

Achieved 99.46% MNIST test accuracy in ~2 minutes using a custom Julia MLP with data augmentation and momentum, demonstrating efficient CPU/GPU implementation with minimal libraries.

Workflow

• Learn ML concepts and Julia syntax
• Derive MLP forward and back‑propagation equations
• Implement core math functions in Julia (CPU and GPU)
• Add data augmentation (random distortions)
• Train MLP on augmented MNIST with momentum
• Iterate hyperparameters (hidden size, batch size, activation)
• Measure performance and accuracy
• Document results

Evidence

I finally reached the goals I'd set of at least 99% accuracy on the training dataset and no greater than 5 minutes of training time. In fact, I didn't just meet those goals, I surpassed them. Not only did I achieve 99% accuracy on the training set (and validation set, I might add) I eventually achieved at least 99% accuracy on the MNIST Test set for a final accuracy of 99.46%, and I did that in a training time of about 2 minutes, which is more than twice as fast as my initial goal.

Source