layernorm
Here are 10 public repositories matching this topic...
Code for the paper "On the Expressivity Role of LayerNorm in Transformers' Attention" (Findings of ACL'2023)
-
Updated
Sep 27, 2024 - Python
Implement layer normalization GRU in pytorch
-
Updated
Apr 8, 2026 - Python
WGAN with feedback from discriminator& LayerNorm instead of BatchNorm
-
Updated
Sep 12, 2020 - Python
LLM pretraining from scratch on FineWeb dataset (architecture and all components explained), plus optimal use of GPU on SLURM cluster
-
Updated
May 12, 2026 - Python
Fused normalisation CUDA kernels for PyTorch — residual-add+LayerNorm/RMSNorm, fp8 outputs, fast deterministic backward, torch.compile-native, with reproducible A100/H100 benchmarks
-
Updated
Aug 25, 2026 - Python
Topology and geometry of Transformer hidden states via persistent homology. The manifold is a 1D arc (PC1=norm, r=0.999), not a torus or sphere. PC2=position, PC3=surprisal+POS. Validated up to 40B.
-
Updated
Apr 15, 2026 - Python
Governed RMSNorm/LayerNorm universal kernel — correctness-verified normalization with optional SHA3-256 hash-chained provenance receipts. Mirrors HF SZLHOLDINGS/szl-governed-norm. Λ = Conjecture 1 (advisory) · Apache-2.0.
-
Updated
Aug 8, 2026 - Python
Geometric phase structure in Transformer hidden states. LayerNorm placement predicts manifold geometry — 6x difference in PCA concentration. 9 models, 13 experiments.
-
Updated
Mar 30, 2026 - Python
Improve this page
Add a description, image, and links to the layernorm topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the layernorm topic, visit your repo's landing page and select "manage topics."