# LLMs: Sources Hub: [[Topics/LLMs/LLMs|LLMs]] | Source | Link | Supports | |---|---|---| | He, Zhang, Ren and Sun, "Deep Residual Learning for Image Recognition", arXiv 1512.03385 (Dec 2015), CVPR 2016 | [link](https://arxiv.org/abs/1512.03385) | Residual blocks, "two or three layers", degradation problem, "unlikely to be caused by vanishing gradients", 12 pages, 20/56 and 18/34-layer plain nets, Highway Nets cited | | He, Zhang, Ren and Sun, "Identity Mappings in Deep Residual Networks", ECCV 2016 | [link](https://arxiv.org/abs/1603.05027) | Pre-activation residual unit | | Pearson, Ledford, Hutson and Van Noorden, "Exclusive: the most-cited papers of the twenty-first century", Nature, Apr 2025 | [link](https://www.nature.com/articles/d41586-025-01125-9) | ResNet ranked #1 | | Welch Labs, "The most cited paper of the century is a brilliant hack", YouTube, Sep 1, 2026 | [link](https://www.youtube.com/watch?v=QgH9sr7G13Q) | Source of the reposted video and caption; 74- vs 8-layer demo (38.9% → 72.6%) | | Balduzzi et al., "The Shattered Gradients Problem", ICML 2017 | [link](https://arxiv.org/abs/1702.08591) | Shattered gradients (2017) | | Srivastava, Greff and Schmidhuber, "Highway Networks", arXiv May 2015 | [link](https://arxiv.org/abs/1505.00387) | Gated skip connections; priority | | Veit, Wilber and Belongie, "Residual Networks Behave Like Ensembles of Relatively Shallow Networks", NeurIPS 2016 | [link](https://arxiv.org/abs/1605.06431) | Lesion study; 110-layer CIFAR ResNet; VGG drops to chance | | Darcet, Oquab, Mairal and Bojanowski, "Vision Transformers Need Registers", ICLR 2024 | [link](https://arxiv.org/abs/2309.16588) | Artifact tokens; register tokens as the fix | | Burtsev et al., "Memory Transformer", 2020 | [link](https://arxiv.org/abs/2006.11527) | Earlier memory/extra tokens idea | | Elhage et al., "A Mathematical Framework for Transformer Circuits", Anthropic, Dec 2021 | [link](https://transformer-circuits.pub/2021/framework/index.html) | "Residual stream" term | | Vaswani et al., "Attention Is All You Need", 2017 | [link](https://arxiv.org/abs/1706.03762) | Residual connection around each sub-layer | | Rombach et al., "High-Resolution Image Synthesis with Latent Diffusion Models", 2022 | [link](https://arxiv.org/abs/2112.10752) | Stable Diffusion U-Net built from ResNet blocks | | Li et al., "Visualizing the Loss Landscape of Neural Nets", NeurIPS 2018 | [link](https://arxiv.org/abs/1712.09913) | Loss-landscape method; skips smooth it | | He et al., "Delving Deep into Rectifiers" (PReLU), 2015 | [link](https://arxiv.org/abs/1502.01852) | 4.94% top-5, first claim of beating human-level ImageNet | | Krizhevsky, Sutskever and Hinton, "ImageNet Classification with Deep Convolutional Neural Networks", NeurIPS 2012 | [link](https://papers.nips.cc/paper/2012/hash/c399862d3b9d6b76c8436e924a68c45b-Abstract.html) | AlexNet 15.3% top-5 (not human-level) | | Dinh et al., "Sharp Minima Can Generalize For Deep Nets", 2017 | [link](https://arxiv.org/abs/1703.04933) | Flat-minima claim is contested | | Schaeffer, Miranda and Koyejo, "Are Emergent Abilities of Large Language Models a Mirage?", 2023 | [link](https://arxiv.org/abs/2304.15004) | Emergence is contested | | 3Blue1Brown neural-network and GPT series | [link](https://www.3blue1brown.com/topics/neural-networks) | Source of the uncredited Oct 14 article | | @0xDeliriumm, ResNet post, Oct 24, 2026 | [link](https://x.com/0xDeliriumm/status/2113976535770071470) | Seed post | | @0xDeliriumm, "What LLM visualization documents reveal about how AI thinks", Oct 14, 2026 | [link](https://x.com/0xDeliriumm/status/2110472580679258160) | Quoted article |