The Llama Family (Meta), and What Came After¶

In one line: Llama made "download a strong model and run it yourself" normal between 2023 and 2025, but its last release was Llama 4 (April 2025); in 2026 Meta moved its frontier work to a closed Muse line and returned to open weights only with the smaller Muse Glimmer - so "Meta = open" is no longer a safe assumption. Last reviewed: 2026-09-30

The short version¶

• Llama was the open-weight default. From LLaMA (February 2023) through Llama 3.3 (December 2024), each release gave developers weights they could download, fine-tune and run on their own hardware, and most of the open-model ecosystem was built on them.
• "Open weights", not "open source". The weights are downloadable, but the licence restricts use: companies above 700 million monthly active users need Meta's permission, derived models must carry the "Llama" name, and the multimodal Llama models exclude EU-based companies and individuals from the licence grant. The training data was never released.
• The design was deliberately conservative until Llama 4. Llama 1 to 3 were dense decoder-only transformers that won by training small models on far more data than compute-optimal rules suggested. Llama 4 (April 2025) switched to mixture-of-experts and native multimodality.
• Llama 4 was the last Llama. Its largest model, Behemoth, was previewed but never released. Meta formed Meta Superintelligence Labs in 2025, and in April 2026 launched Muse Spark, a closed model, as the new engine of Meta AI.
• Open weights returned in a different form. In August 2026 Meta released Muse Glimmer, a 30B model under Apache 2.0, and said an open-weight version of Muse Spark 1.2 was coming. As of September 30, 2026 that Spark release has not been published on Meta's channels.

Release timeline¶

"Summary" links go to paper summaries in this repo where one exists.

Llama¶

After Llama: the Muse line¶

The through-line¶

1. Small models, lots of data¶

The founding idea of LLaMA was to spend compute on data, not parameters. Chinchilla had worked out the compute-optimal ratio of model size to training tokens for a fixed training budget. LLaMA deliberately trained past that point, because a model that is used a lot is cheaper to serve when it is small:

Chinchilla question:  best model for a fixed TRAINING budget?
LLaMA question:       best model for a fixed INFERENCE budget?
                      -> smaller model, many more tokens
LLaMA 1:  up to 1.4T tokens    Llama 2:  2T    Llama 3:  ~15T    Llama 4:  30T+

This is why a 7B or 8B Llama could run on a laptop and still be useful, and why "Llama-sized" became shorthand for what open models could do.

2. Standard parts, carefully assembled¶

Llama 1 to 3 introduced little new architecture. They combined proven parts: RoPE position embeddings, RMSNorm, SwiGLU activations, and from Llama 2 70B onward grouped-query attention to shrink the KV cache (see the KV cache explainer). Post-training moved from RLHF in Llama 2 toward DPO-style preference optimisation and teacher-generated data in Llama 3.x. Llama 3.3 is the clearest case: a 70B model trained with help from the 405B model as a teacher, a form of knowledge distillation.

3. Llama 4: a change of direction¶

Llama 4 broke with the conservative recipe in three ways: mixture-of-experts (only a fraction of parameters runs per token), early-fusion multimodality (image and text tokens in one model from the start of training), and a very long context claim for Scout (10M tokens, from 256K in pre- and post-training). The launch was also controversial: LMArena said the Maverick version Meta submitted to its leaderboard was an experimental chat-tuned variant that differed from the public release. See human preference arenas and contamination and saturation for why that matters.

4. Safety as a toolkit, not only a model¶

Because anyone could fine-tune the weights, Meta could not rely on the model alone to behave. It released separate safety components: Llama Guard (an input and output classifier), and an acceptable use policy attached to the licence. This "ship the guardrails alongside the model" approach is now common in open-weight deployments; see the hands-on guardrails and safety page.

5. The 2026 pivot¶

With Muse Spark, Meta put its frontier model behind its own apps and an API, like OpenAI, Anthropic and Google. Meta said at the time that its current Llama models would remain available. Four months later it returned to open weights with Muse Glimmer, a model sized for a single consumer GPU under a standard Apache 2.0 licence rather than a custom Llama licence. The resulting shape resembles Google's: a closed frontier line plus a smaller open line (compare Gemini and Gemma).

Open vs closed¶

The Open Source Initiative does not consider the Llama licences open source, because of the use restrictions and because training data is not released. The more precise term is "open weights". This distinction is the subject of open vs closed weights.

How to read the naming¶

• Llama generations. "LLaMA" (with that capitalisation) was the 2023 original. From Llama 2 it is written "Llama". Point releases (3.1, 3.2, 3.3) added sizes or modalities to the same generation rather than retraining everything.
• Sizes in the name. Up to Llama 3.3, the suffix is the parameter count (Llama-3.1-8B, -70B, -405B). "Instruct" or "Chat" marks the instruction-tuned version; without it you have the base model.
• Llama 4 animal names. Scout, Maverick and Behemoth are sizes. The official model IDs spell out active parameters and expert count, for example Llama-4-Maverick-17B-128E-Instruct (17B active, 128 experts).
• Muse. Muse is the new family. "Spark" is the closed frontier line with point versions (1.1, 1.2); "Glimmer" is the small open model; "Muse Image" is image generation; "Muse" is also the name of Meta's consumer AI agent. Expect this scheme to settle as more models ship.

Why Llama mattered even if it stops¶

• It showed that a strong model could be released openly without the predicted immediate harms, which shifted the policy debate (see US AI policy).
• It created the tooling ecosystem: llama.cpp, Ollama, vLLM, LoRA fine-tunes and quantised builds grew up around Llama weights. See LoRA, QLoRA, GPTQ and AWQ quantization, vLLM, and the open-source stack explainer.
• Its gap is now being filled largely by Chinese labs; see DeepSeek and Qwen, and by Mistral and Google's Gemma.

What to watch¶

• Whether the open Muse Spark 1.2 weights ship. This is the single clearest test of Meta's renewed open-weights commitment. As of September 30, 2026 they have been announced but not released.
• Licence choice. Muse Glimmer uses Apache 2.0, with none of the Llama licence's MAU cap or EU carve-out. Whether larger open Muse models keep that licence or return to Llama-style terms will decide how widely they are adopted.
• The fate of existing Llama models. Meta says they remain available. Watch for deprecations on cloud platforms and whether Meta keeps publishing Llama safety tools.
• Meta's compute efficiency claim. Meta says Muse Spark matches Llama 4 Maverick's capabilities with over an order of magnitude less compute. Independent replication or a technical report would make this checkable; see cost of training.

Read next¶

• Paper summaries: LLaMA, Llama 2, Llama 3.3, Llama 4, Llama Guard, Chinchilla, Grouped-query attention, RoPE, YaRN
• Other families: GPT and o-series, Claude, Gemini, DeepSeek, Qwen, Mistral
• Explainers: Open vs closed weights, Open-source stack, Labs landscape
• Hands-on: Fine-tuning vs RAG, Quantization and distillation, Inference servers

Sources¶

• Meta, "The Llama 4 herd" (April 5, 2025; parameters, experts, context, 30T+ tokens, Behemoth still training): https://ai.meta.com/blog/llama-4-multimodal-intelligence/
• Meta, Llama 4 Community License Agreement (700M MAU clause, "Built with Llama", "Llama" name prefix): https://dev.meta.ai/llama/llama4/license/
• Meta, Llama 4 Acceptable Use Policy (EU restriction for multimodal models): https://dev.meta.ai/llama/llama4/use-policy/
• Meta, "Introducing Llama 3.1": https://ai.meta.com/blog/meta-llama-3-1/
• Meta, "Llama 3.2: Revolutionizing edge AI and vision with open, customizable models": https://ai.meta.com/blog/llama-3-2-connect-2024-vision-edge-mobile-devices/
• Meta, "Introducing Meta Llama 3": https://ai.meta.com/blog/meta-llama-3/
• Meta, "Introducing Code Llama": https://ai.meta.com/blog/code-llama-large-language-model-coding/
• Touvron et al., "LLaMA: Open and Efficient Foundation Language Models": https://arxiv.org/abs/2302.13971
• Touvron et al., "Llama 2: Open Foundation and Fine-Tuned Chat Models": https://arxiv.org/abs/2307.09288
• Meta, "Introducing Muse Spark: Scaling Towards Personal Superintelligence" (April 8, 2026): https://ai.meta.com/blog/introducing-muse-spark-msl/ and https://about.fb.com/news/2026/04/introducing-muse-spark-meta-superintelligence-labs/
• Meta, "Introducing Muse Spark 1.1" (July 9, 2026; 1M context, Meta Model API public preview): https://ai.meta.com/blog/introducing-muse-spark-meta-model-api/
• Hugging Face, "Meta is back with Muse Glimmer" (August 10, 2026; 30B dense, Apache 2.0; the model card gives 131,072+ tokens of context): https://huggingface.co/blog/muse-glimmer and model card https://huggingface.co/meta-models/Muse-Glimmer-30B
• Alexandr Wang on X (August 10, 2026), open weights for Muse Glimmer and "a version of muse spark 1.2 coming soon": https://x.com/alexandr_wang/status/2086755368596902004
• VentureBeat, "Goodbye, Llama? Meta launches new proprietary AI model Muse Spark" (Meta spokesperson: current Llama models stay available): https://venturebeat.com/technology/goodbye-llama-meta-launches-new-proprietary-ai-model-muse-spark-first-since
• Wikipedia, "Llama (language model)" (cross-check of release dates, OSI position, LMArena controversy): https://en.wikipedia.org/wiki/Llama_(language_model)