Quick Reference Guide - All Papers at a Glance¶

One line on what each of the 144 papers contributed and why it mattered, grouped by category and ordered by paper number. This is the lookup table; for a card view with relevance badges see BROWSE.md, for the generated listings see INDEX.md and TAGS.md, and for a guided order see ROADMAP.md.

Architecture Papers¶

#	Paper	Year	Key Contribution	Impact
1	Attention Is All You Need	2017	Self-attention and the Transformer; no recurrence, fully parallel	The architecture underneath essentially every model in this collection
11	Vision Transformer (ViT)	2020	Treat image patches as a token sequence	Removed the CNN prior; the vision encoder in modern multimodal models
20	Mamba	2023	Selective state-space model with linear-time sequence scaling	The most credible non-attention alternative; now used in hybrids
37	Mixtral & Mixture-of-Experts	2024	Sparse mixture of experts, top-2 of 8 routing	MoE became the default frontier architecture
55	Seq2Seq	2014	LSTM encoder-decoder mapping variable-length sequences	Established encoder-decoder; its bottleneck motivated attention
66	Bahdanau Attention	2014	Learned soft alignment over all encoder states	Invented attention; direct ancestor of self-attention
67	Switch Transformer	2021	Top-1 expert routing at trillion-parameter scale	Made sparse MoE stable enough to actually train
73	ResNet	2015	Identity skip connections make very deep networks trainable	Residual connections sit in every Transformer block
74	U-Net	2015	Symmetric encoder-decoder with skip connections	The denoiser backbone of the entire diffusion era
75	Grouped-Query Attention	2023	Share key/value heads across groups of query heads	Shrinks the KV cache; standard in modern LLM serving
88	Masked Autoencoders (MAE)	2021	Mask 75% of patches, reconstruct with an asymmetric decoder	Scalable self-supervised pretraining for vision
131	Longformer	2020	Combined local windowed attention with task-specific global attention for linear cost	Established the local-plus-global pattern used in many efficient Transformers
141	Multi-head Latent Attention (DeepSeek-V2)	2024	Compressed keys and values into a cached low-rank latent with decoupled RoPE	The attention design of DeepSeek-V3/R1 and other open models

Language Model Papers¶

#	Paper	Year	Key Contribution	Impact
3	BERT	2018	Bidirectional masked-language-model pretraining	Made pretrain-then-fine-tune standard; still runs in search and retrieval
4	GPT-3	2020	175B parameters; in-context few-shot learning	Prompting displaced fine-tuning; the direct run-up to ChatGPT
5	InstructGPT (RLHF)	2022	Reinforcement learning from human feedback on a base model	Turned raw LLMs into assistants; the recipe behind ChatGPT and Claude
14	Constitutional AI	2022	Model critiques itself against written principles (RLAIF)	Cut human labelling out of alignment; the method behind Claude
15	LLaMA	2023	Compute-optimal training; smaller models trained far longer	Kicked off the open-weight ecosystem
17	LLaMA 2	2023	Open weights with a fully documented RLHF chat pipeline	Made commercially usable open chat models mainstream
19	DPO	2023	Optimise preferences directly, with no reward model or RL loop	Replaced PPO-based RLHF for most open fine-tunes
26	DeepSeek-R1	2025	Reasoning learned by pure RL on verifiable rewards	Open model matching o1, and published what OpenAI kept closed
27	DeepSeek-V3	2024	671B MoE, 37B active, trained for under $6M	Frontier quality at a fraction of Western training budgets
28	Qwen3	2025	Hybrid thinking / non-thinking modes in a single model	Leading open-weight multilingual family
30	Claude 3.5 Sonnet	2024	Computer use: driving a GUI from screenshots	Opened the desktop-automation agent category
31	OpenAI o1	2024	RL training to spend inference compute on hidden reasoning	Created the reasoning-model category
33	LLaMA 3.3	2024	Distillation and post-training recover 405B quality at 70B	Frontier-class open models on modest hardware
36	GPT-4	2023	Large multimodal model with predictable scaling, disclosure withheld	The capability jump that made LLMs a general product platform
40	GPT-4o	2024	One network natively trained across text, audio and vision	Made real-time conversational voice practical
41	Llama 4	2025	Open-weight MoE, natively multimodal, 10M-token context	Brought frontier architecture choices into open weights
42	GPT-5	2025	A unified model routing between fast answers and deep reasoning	Collapsed the confusing model-picker era into one system
43	Claude 4 Family	2025	Extended thinking plus long-horizon agentic tool use	State of the art on real software-engineering work
56	Codex	2021	GPT fine-tuned on public code; HumanEval and pass@k	Produced GitHub Copilot and the whole code-model lineage
64	GPT-2	2019	A 1.5B-parameter LM performing tasks zero-shot	Established scale-plus-prompting, and the staged-release precedent
65	T5	2019	Cast every NLP task as text-to-text, with systematic ablations	Standardised the framing; its encoder still powers image models
93	GPT-1	2018	Generative pretraining, then discriminative fine-tuning	The original GPT recipe, at 117M parameters
94	PaLM	2022	540B dense model trained across two TPU pods via Pathways	The practical ceiling for dense models; strong CoT results
95	Mistral 7B	2023	GQA and sliding-window attention in a small Apache-2.0 model	Made open weights genuinely competitive
96	Llama Guard	2023	An LLM fine-tuned to classify prompt and response safety	The open reference for production content moderation
135	phi-1 (Textbooks Are All You Need)	2023	Trained a small code model on filtered web code plus synthetic textbooks and exercises	Started the phi small-model line and the synthetic-pretraining-data trend

Image, Video & 3D Generation Papers¶

#	Paper	Year	Key Contribution	Impact
2	GANs	2014	Generator and discriminator trained against each other	Launched modern generative modelling; dominant for images until diffusion
6	Diffusion Models (DDPM)	2020	Learn to reverse a fixed noising process	Made diffusion practical and displaced GANs for image generation
7	Stable Diffusion	2022	Run diffusion in a compressed VAE latent space	10-100x cheaper generation; put text-to-image on consumer GPUs
44	Sora & Diffusion Transformers	2022-2024	Replace the U-Net with a Transformer; video as spacetime patches	Launched text-to-video and the DiT backbone
48	DALL-E 3	2023	Retrain on synthetic, highly descriptive captions	Fixed prompt following and text rendering in images
57	Variational Autoencoder (VAE)	2013	Amortised variational inference via the reparameterisation trick	Gave generative models a usable latent space; the encoder in latent diffusion
69	Classifier-Free Guidance	2021	Interpolate conditional and unconditional predictions	The guidance-scale dial and negative prompts in every image model
70	DDIM	2020	Deterministic non-Markovian sampling of a trained diffusion model	Cut sampling from 1000 steps to ~20 with no retraining
71	ControlNet	2023	A trainable encoder copy injecting spatial conditioning	Precise pose, depth and edge control over image generation
72	Flow Matching / SD3	2022-2024	Learn straight probability paths instead of a noising chain	The formulation behind SD3 and Flux
89	VQ-VAE	2017	Quantise latents against a learned codebook	Made images and audio tokenisable for autoregressive models
90	VQ-GAN	2020	VQ-VAE tokens with adversarial and perceptual losses	The compress-then-model recipe behind high-resolution synthesis
91	Imagen	2022	A frozen large text encoder plus cascaded diffusion	Showed text understanding, not the image model, was the bottleneck
92	DreamBooth	2022	Bind a rare token to a subject with a prior-preservation loss	Created the personalisation and fine-tune-sharing ecosystem
117	NeRF	2020	Represented a scene as a neural radiance field rendered by differentiable volume rendering	Launched neural rendering; the base of modern 3D reconstruction research
118	3D Gaussian Splatting	2023	Replaced neural fields with explicit anisotropic Gaussians rasterised in real time	Became the dominant practical 3D capture and rendering method
119	DALL-E 2 (unCLIP)	2022	Generated images by decoding CLIP image embeddings produced from text by a learned prior	The 2022 model that brought text-to-image to the public
120	Consistency Models	2023	Trained models to map any point on a diffusion trajectory directly to its clean endpoint	Opened the door to one- and few-step generation

Multimodal & Audio Papers¶

#	Paper	Year	Key Contribution	Impact
8	CLIP	2021	Contrastive image-text pretraining on 400M pairs	Zero-shot classification, and the text conditioning inside image generators
23	GPT-4V(ision)	2023	Production vision-language system card	Brought image understanding into mainstream assistant products
29	Gemini 2.5	2025	Long-context multimodal reasoning with thinking budgets	Pushed production context windows past 1M tokens
32	SAM 2	2024	Promptable segmentation unified across images and video	Foundation-model segmentation with memory across frames
46	LLaVA	2023	Connect a frozen vision encoder to an LLM with a projection layer	Made open multimodal models buildable by anyone
47	Gemini 3	2025	Frontier multimodal reasoning with a Deep Think mode	First model past 1500 LMArena Elo
49	Whisper	2022	Weakly supervised ASR on 680k hours across 99 languages	The default open speech model; robust with no fine-tuning
121	AudioLM	2022	Modelled audio as a hierarchy of semantic and acoustic tokens with language models	The template for token-based audio and music generation
122	VALL-E	2023	Treated text-to-speech as language modelling over neural codec tokens	Zero-shot voice cloning from a short sample; raised the deepfake-voice alarm

Technique & Method Papers¶

#	Paper	Year	Key Contribution	Impact
9	Chain-of-Thought	2022	Prompt the model to show intermediate steps	Unlocked multi-step reasoning; ancestor of every reasoning model
10	LoRA	2021	Freeze the base model, train low-rank update matrices	Made fine-tuning affordable; the default adapter format
12	Scaling Laws	2020	Loss falls as a power law in compute, data and parameters	Made frontier training budgets predictable rather than speculative
13	RAG	2020	Retrieve documents at inference and condition generation on them	The default way production systems ground answers in private data
16	FlashAttention	2022	IO-aware tiled attention that never materialises the N x N matrix	Longer contexts and faster training on the same hardware
18	Chinchilla	2022	For a fixed compute budget, scale data alongside parameters	Corrected GPT-3-era over-parameterisation; every lab retuned to it
21	ReAct	2023	Interleave reasoning traces with tool-use actions	The loop underneath most LLM agent frameworks
22	QLoRA	2023	LoRA on top of a 4-bit frozen base model	65B fine-tuning on a single consumer GPU
24	Toolformer	2023	Model teaches itself when to call APIs, self-supervised	Early proof that tool use can be learned rather than prompted
25	Tree of Thoughts	2023	Search over branching reasoning paths with backtracking	Generalised CoT into deliberate search; precursor to test-time compute
34	Meta-CoT	2025	Model the search process itself, not just the final chain	Theoretical framing for System 2 reasoning in LLMs
35	rStar-Math	2025	Small models self-evolve via MCTS with process rewards	7B models matching o1-preview on competition maths
38	GRPO	2024	Group-relative advantages replace PPO's learned critic	Made large-scale reasoning RL cheap enough to run; powers R1
39	RLVR	2024-2025	Reward verifiable correctness instead of learned human preference	The training signal behind the reasoning-model generation
45	Speculative Decoding	2022	A draft model proposes tokens, the target model verifies	2-3x faster serving with mathematically identical outputs
50	Scaling Test-Time Compute	2024	Trade inference compute against parameter count, optimally	The theory that justified o1-style reasoning models
51	Process Reward Models	2023	Supervise each reasoning step, not just the final answer	How reasoning models verify their chains; PRM800K released
52	PagedAttention & vLLM	2023	Paged, non-contiguous KV-cache allocation	24x serving throughput; vLLM is the industry default
53	Word2Vec	2013	Dense embeddings learned from context prediction	Started representation learning for language
54	RoPE	2021	Encode relative position by rotating queries and keys	The positional scheme in nearly every modern LLM
58	Generative Agents	2023	Memory stream plus reflection and planning	The reference architecture for long-running agent simulations
59	Model Context Protocol (MCP)	2024	An open client-server protocol for model-tool integration	Collapsed M x N custom integrations; adopted industry-wide
60	GraphRAG	2024	LLM-extracted knowledge graph with community summaries	Answers corpus-level questions that vector RAG cannot
61	AlphaGeometry	2024	Neuro-symbolic prover: LM proposes constructions, solver deduces	Olympiad-medallist geometry with no human proof data
62	AlphaEvolve	2025	LLM-guided evolutionary search over programs	Beat a 56-year-old matrix-multiplication record; deployed at Google
63	PPO	2017	Clipped surrogate objective for stable policy updates	The RL algorithm inside RLHF
68	AlphaFold 2	2021	Evoformer plus a structure module over multiple sequence alignments	Solved protein structure prediction; 200M+ structures released
76	ZeRO & Megatron-LM	2019	Shard optimiser state, gradients and parameters across devices	How trillion-parameter training physically happens
77	Self-Consistency	2022	Sample many chains and take the majority answer	The cheapest reliable accuracy gain on reasoning tasks
78	Reflexion	2023	Agent writes verbal self-critique into episodic memory	Let agents learn from failure without weight updates
79	Self-Instruct	2022	Bootstrap instruction data from the model itself	Made instruction tuning cheap; produced Alpaca and its descendants
80	FLAN	2021	Multi-task instruction fine-tuning for zero-shot transfer	Established instruction tuning as a standard training stage
81	Emergent Abilities	2022-2023	Do capabilities appear discontinuously with scale?	Paired with the Mirage rebuttal; a lesson in metric choice
82	Sparse Autoencoders	2022-2024	Decompose activations into sparse, interpretable features	The leading interpretability method; enabled feature steering
83	Sleeper Agents	2024	Backdoors that survive safety fine-tuning	Showed current alignment training can fail to remove deception
84	SWE-bench	2023	Resolve real GitHub issues in real repositories	The benchmark frontier coding agents are measured on
85	LLM-as-a-Judge	2023	Use a strong model as evaluator; MT-Bench and Chatbot Arena	How most model comparisons are now run
86	GPTQ & AWQ	2022-2023	Post-training 4-bit weight quantisation	70B models on one consumer GPU; why local inference exists
87	Dense Retrieval	2019-2020	DPR, ColBERT and Sentence-BERT embedding retrieval	The retriever layer under every RAG system
97	STaR	2022	Keep self-generated rationales that reach the right answer, retrain	The self-improvement loop behind modern reasoning training
98	Quiet-STaR	2024	Learn latent rationales before every token	Generalised STaR to arbitrary text; predates o1
99	Self-Refine	2023	The same model generates, critiques and revises	The LLM-as-critic pattern used across agents and reasoning
100	Voyager	2023	Lifelong agent that writes and stores reusable skills as code	The first convincing open-ended LLM agent
101	AlphaFold 3	2024	Diffusion-based prediction over proteins, DNA, RNA and ligands	Extended structure prediction to biomolecular complexes
102	AlphaZero	2017	Self-play RL with MCTS from the rules alone	The ancestor of every self-improvement training loop
103	KTO	2024	Prospect-theoretic loss on unpaired binary feedback	Alignment without expensive paired preference data
104	Genie	2024	Latent action model learned unsupervised from video	First foundation world model; playable worlds from one image
105	DreamerV3	2023	Model-based RL that learns inside an imagined world model	One hyperparameter set across 150+ tasks; Minecraft diamonds
106	ESM-2 / ESMFold	2023	Protein language model predicting structure without MSAs	Scaling laws transfer to biology; 617M structures released
107	CICERO	2022	Language model fused with a strategic planning engine	Human-level Diplomacy, negotiation included
126	Induction Heads	2022	Identified induction heads and tied them to an abrupt jump in in-context learning	A founding result of mechanistic interpretability
127	GCG Adversarial Attacks	2023	Gradient-guided search for universal adversarial suffixes against aligned LLMs	Showed refusal training is brittle; spurred filters, smoothing and classifier defences
128	Weak-to-Strong Generalization	2023	Built a testable analogy for superhuman oversight: weak models supervising strong ones	Made scalable-oversight research empirical
129	Red Teaming LMs	2022	Used language models, from prompting to RL, to generate red-team test cases	Template for automated pre-release safety testing
130	YaRN	2023	Extended RoPE context windows with frequency-aware interpolation and attention temperature scaling	The standard way open models reach long context
132	RULER	2024	A configurable long-context benchmark measuring effective rather than advertised context	Standard long-context report card in model releases
133	FineWeb	2024	Released a 15T-token web corpus with ablations for every processing step	Made pretraining data curation reproducible research
134	Knowledge Distillation	2015	Trained small students on a teacher's temperature-softened output distribution	How the industry ships small models, from DistilBERT to R1-distilled models
136	BPE Subword Units	2015	Adapted byte pair encoding to learn open-vocabulary subword units	The basis of nearly every LLM tokenizer
137	MMLU	2020	A 57-subject multiple-choice test of knowledge and problem solving	The headline LLM benchmark of 2021-2024; now saturated
138	ARC-AGI	2019	Defined intelligence as efficiency of skill acquisition and built the ARC benchmark to test it	The reasoning benchmark frontier models are still measured against
139	OSWorld	2024	An execution-graded benchmark of agents operating real desktop applications	The standard computer-use agent benchmark
140	Model Collapse	2024	Showed recursive training on generated data progressively loses distribution tails	Framed the synthetic-data and data-provenance debate
142	Adam	2014	Combined momentum with per-parameter adaptive learning rates and bias correction	The optimiser behind almost every model in this collection
143	Mixed Precision Training	2017	Trained in half precision with FP32 master weights and loss scaling	Standard practice for all large training; led on to BF16 and FP8
144	Muon	2024	Orthogonalised momentum updates for matrix parameters, scaled to LLM training	Adopted for frontier-scale open models such as Kimi K2
145	Measuring AI Ability to Complete Long Tasks	2025	Defined the 50% time horizon: the human task length a model completes half the time, and its doubling trend	The standard capability-trend chart for agents and forecasting
146	Deep RL from Human Preferences	2017	Learned a reward model from human comparisons of short clips, then ran deep RL on it	The origin of RLHF, leading to InstructGPT and chat assistants
147	Learning to Summarize from Human Feedback	2020	Applied the human-preference reward-model loop to language and beat supervised baselines on summarization	The direct template for InstructGPT
148	Tulu 3	2024	A fully open SFT, DPO and RLVR post-training recipe that gave RLVR its name	A reproducible baseline for open post-training; made RLVR a standard stage
149	Why Language Models Hallucinate	2025	Explained hallucination statistically and showed that accuracy-only grading rewards guessing	Reframed hallucination as an evaluation-design problem
150	On the Dangers of Stochastic Parrots	2021	Argued that ever-larger language models carry environmental, data and social risks	Set the terms of the critical side of the scaling debate
151	Circuit Tracing and the Biology of an LLM	2025	Cross-layer transcoders and attribution graphs that trace the steps a model takes on a single prompt	Causal evidence of multi-hop reasoning, planning and hallucination circuits
152	Reasoning Models Don't Always Say What They Think	2025	Hint-based test of chain-of-thought faithfulness in reasoning models	Showed CoT monitoring is useful but insufficient as a safety layer
153	Emergent Misalignment	2025	Showed finetuning on insecure code alone makes models broadly misaligned, with controls ruling out a jailbreak	Opened a research thread on how narrow training shifts a model's whole persona
154	s1: Simple Test-Time Scaling	2025	1,000-example s1K dataset plus budget forcing give a competitive open reasoning model	A cheap baseline; thinking-length control became common
155	DAPO	2025	Open large-scale RL system with four fixes to GRPO-style training	A reproducible reference for reasoning RL beyond closed labs
156	Mamba-3	2026	Next-generation state space layer from the Mamba authors	Candidate layer for future hybrid open-weight models

Robotics & Embodied AI Papers¶

#	Paper	Year	Key Contribution	Impact
123	RT-2	2023	Fine-tuned web-scale vision-language models to output robot actions as tokens	Created the VLA model class behind today's robot foundation models
124	pi0	2024	Paired a vision-language model with a flow-matching action expert that generates continuous action chunks	A widely used open robot foundation model
125	Open X-Embodiment	2023	Pooled robot data from 34 labs and showed positive transfer across robot bodies	Default pretraining pool for open robot models; ICRA 2024 best paper

Essays & Landmark Posts¶

#	Paper	Year	Key Contribution	Impact
108	Computing Machinery and Intelligence	1950	Replaced 'can machines think?' with a behavioural test, the imitation game	Set the terms of the AI debate for 75 years; still the reference point for human-likeness tests
109	The Unreasonable Effectiveness of RNNs	2015	Showed small character-level RNNs learning structure from raw text	Popularised generative language modelling years before GPT
110	Software 2.0	2017	Argued neural networks are a new programming paradigm: specify data and goals, let optimisation write the program	Shaped how engineers think about ML systems; extended later as 'Software 3.0'
111	The Bitter Lesson	2019	70 years of AI show general methods that scale with compute beat human-knowledge engineering	The most-cited short argument for scaling; a lens for reading every paper here
112	The Scaling Hypothesis	2020	Interpreted GPT-3 as evidence that scale alone keeps producing new capabilities	Articulated the bet frontier labs then made with their capital
113	Situational Awareness	2024	Extrapolated compute and algorithmic trends into an AGI-by-2027 forecast and a US security agenda	Widely read in policy circles; defined vocabulary for the AI-race debate
114	Machines of Loving Grace	2024	Laid out specific ways powerful AI could accelerate biology, neuroscience, development and governance	The most detailed benefits case from a frontier-lab leader
115	Building Effective Agents	2024	Named the core agentic patterns (chaining, routing, parallelisation, orchestrator-workers, evaluator-optimizer)	The practical playbook most agent builders now start from
116	Welcome to the Era of Experience	2025	Argued the next leap comes from agents learning from their own experience rather than human data	A counterweight to pure-scaling views from two RL pioneers

Browsing by Topic or Year¶

These views are generated from papers.json, so they never go stale:

By topic - TAGS.md groups every paper under its topic tags (retrieval, alignment, efficiency, agents, reasoning, interpretability, and the rest).
By year - the By Year block in the README is regenerated on every build; INDEX.md has the full clickable list by category.

This page deliberately does not repeat those groupings by hand - a second, hand-typed copy is the thing that drifts.

Numbers Worth Remembering¶

Figures taken from the summaries themselves, not from memory.

Paper	Scale	Note
BERT	110M / 340M params	Base and Large; tiny by current standards
GPT-1	117M params	Where the recipe starts
GPT-2	1.5B params	The "too dangerous to release" model
GPT-3	175B params, 300B tokens	Under-trained by Chinchilla's rule
Chinchilla	70B params, 1.4T tokens	~20 tokens per parameter; beat 280B Gopher
LLaMA	65B params, 1.4T tokens	Chinchilla-optimal, public data only
PaLM	540B params	Largest dense Transformer of its era
Mixtral	47B total, 13B active	Sparse MoE, top-2 of 8 experts
DeepSeek-V3	671B total, 37B active	Frontier quality under $6M of compute
CLIP	400M image-text pairs	Scraped, not labelled
Whisper	680k hours, 99 languages	Weak supervision at scale

Problem → Solution Mapping¶

Modelling and architecture

Problem	Papers
Sequences can't be processed in parallel	Transformer (1)
The encoder-decoder bottleneck loses information	Bahdanau Attention (66), Seq2Seq (55)
Deep networks won't train	ResNet (73)
Attention is quadratic in sequence length	FlashAttention (16), Mamba (20)
Capacity is too expensive to serve	Mixtral / MoE (37), Switch Transformer (67)
Position needs encoding without hurting extrapolation	RoPE (54)
The KV cache dominates serving memory	GQA (75), PagedAttention (52)

Training and alignment

Problem	Papers
No idea how much compute or data a run needs	Scaling Laws (12), Chinchilla (18)
The model is capable but not helpful	InstructGPT (5), FLAN (80)
RLHF needs a reward model and an RL loop	DPO (19), GRPO (38)
Preference data must be collected in pairs	KTO (103)
Human preference labels are expensive	Constitutional AI (14), RLVR (39)
Instruction data is expensive	Self-Instruct (79)
A model won't fit in memory to train	ZeRO & Megatron-LM (76)
Fine-tuning the whole model costs too much	LoRA (10), QLoRA (22)
The RL algorithm itself is unstable	PPO (63)

Reasoning

Problem	Papers
The model answers hard questions in one shot	Chain-of-Thought (9)
A single chain is unreliable	Self-Consistency (77), Tree of Thoughts (25)
Only the final answer is supervised	Process Reward Models (51)
Reasoning data has to be written by hand	STaR (97), Quiet-STaR (98)
Bigger models are the only way to improve	Test-Time Compute (50), OpenAI o1 (31)
The first draft is wrong	Self-Refine (99), Reflexion (78)

Knowledge, tools and agents

Problem	Papers
The model hallucinates or lacks private data	RAG (13), Dense Retrieval (87)
Retrieval can't answer corpus-level questions	GraphRAG (60)
The model can't act on the world	ReAct (21), Toolformer (24)
Every tool needs a bespoke integration	MCP (59)
Agents forget across long horizons	Generative Agents (58), Voyager (100)

Generation

Problem	Papers
Need photorealistic images	GANs (2), DDPM (6)
Diffusion is too slow and too expensive	Stable Diffusion (7), DDIM (70), Flow Matching (72)
Output ignores the prompt	CFG (69), DALL-E 3 (48), Imagen (91)
Need control over composition	ControlNet (71)
Need a specific person, pet or product	DreamBooth (92)
Images need to be modelled as tokens	VQ-VAE (89), VQ-GAN (90)
Need video, not stills	Sora / DiT (44), Genie (104)

Deployment and evaluation

Problem	Papers
Inference is too slow	Speculative Decoding (45), PagedAttention (52)
The model won't fit on the GPU you have	GPTQ & AWQ (86), QLoRA (22)
Benchmarks don't reflect real work	SWE-bench (84), LLM-as-a-Judge (85)
Benchmark jumps may be an artefact	Emergent Abilities (81)
Nobody knows what the model is doing	Sparse Autoencoders (82)
Safety training may not have worked	Sleeper Agents (83), Llama Guard (96)

Paper Dependencies¶

If a summary assumes something you don't have yet, it is almost always one of these.

Attention (66) ──► Transformer (1) ──► everything else
                        │
                        ├──► BERT (3) ──────────► encoder models, retrieval (87)
                        ├──► GPT-1 (93) ► GPT-2 (64) ► GPT-3 (4) ► GPT-4 (36) ► GPT-5 (42)
                        ├──► ViT (11) ──────────► CLIP (8) ──► LLaVA (46), multimodal
                        └──► MoE (67, 37) ──────► DeepSeek-V3 (27), Llama 4 (41)

Scaling Laws (12) ──► Chinchilla (18) ──► LLaMA (15) ──► Mistral (95), open weights

VAE (57) ──► DDPM (6) ──► Stable Diffusion (7) ──► SD3 / Flow Matching (72)
   │            │              ▲
   │            └──► DDIM (70) │
   └──► VQ-VAE (89) ► VQ-GAN (90)
        U-Net (74) ───────────┘

PPO (63) ──► InstructGPT (5) ──► DPO (19) / KTO (103)
   │                              │
   └──► GRPO (38) ──► RLVR (39) ──┴──► DeepSeek-R1 (26)

CoT (9) ──► Self-Consistency (77) ──► Tree of Thoughts (25) ──► Test-Time Compute (50) ──► o1 (31)
   └──► STaR (97) ──► Quiet-STaR (98)

Read first, in this order, if you are starting cold: Transformer (1) → GPT-3 (4) → InstructGPT (5) → Chain-of-Thought (9) → RAG (13). Everything else has a path back to those five.

When to Reference Each Paper¶

If you are...	Read
Building a chatbot or assistant	4, 5, 14, 13, 43
Adding retrieval over your own documents	13, 87, 60
Fine-tuning on a budget	10, 22, 86
Building an agent that uses tools	21, 24, 59, 78, 100
Serving a model cheaply at scale	52, 45, 86, 75
Training a model from scratch	12, 18, 76, 1, 54
Improving reasoning quality	9, 77, 51, 50, 26
Choosing an alignment method	5, 19, 103, 38, 39
Generating images	6, 7, 69, 70, 71, 92
Generating video	44, 104
Working with speech	49
Building multimodal features	8, 11, 46, 23
Evaluating models honestly	84, 85, 81
Auditing a model for safety	82, 83, 96, 14
Applying this outside language	68, 101, 106, 102, 105, 107

Reading Time¶

Measured from the summaries in this repository, not estimated:


Summaries	107
Words across all summaries	~203,000
Average summary	~1,900 words (about 8-10 minutes)
Whole collection, cover to cover	~17 hours at 200 words/minute

The Quick Stats table in the README carries the canonical figures, including how much source material the summaries compress. Those numbers are regenerated by scripts/build_manifest.py, so they cannot drift from the papers themselves.

Papers: 107 · See also: BROWSE.md · INDEX.md · TAGS.md · ROADMAP.md · COMPARISONS.md