This repository collects several deep learning and NLP implementations I originally developed as separate repositories.
They have now been consolidated here under one roof since they share the same goal:
to practice building core deep learning components from scratch, for learning purposes.
Each subfolder was once its own repo and contains:
- Source code for the implementation
- A dedicated README with explanations and usage examples
⚠️ These projects are educational and not production-ready.
They are focused on learning the mechanics behind modern DL models rather than delivering polished libraries.
Minimal implementation of the Transformer architecture.
Includes:
- Encoding layer
- Attention mechanism
- Simple analysis script
Focus: understanding the Transformer pipeline step by step.
2. S-grad
A scalar autograd engine (inspired by micrograd).
Includes:
engine.py: core automatic differentiationnn.py: simple neural network utilities
Focus: building intuition for backpropagation and how autodiff engines work.
Byte Pair Encoding (BPE) tokenizer implementation.
Includes:
- Core BPE algorithm
- Tokenizer class with encode/decode methods
Focus: understanding subword tokenization in modern NLP pipelines.
Character-level statistical language model using n-grams.
Includes:
- Bi-gram implementation
- N-gram generalization
Focus: exploring classic n-gram models for text generation.
Each subproject is self-contained.
Navigate into any folder and check its README for details:
git clone https://github.com/AbdoAlshoki2/dl-scratch-study.git
cd dl-scratch-study/Naive_Transformer