Skip to content

The ML Engineer Path

In one line: A route for people who want to work below the API - training, fine-tuning, evaluating and serving models themselves - covering the maths you actually need, classical ML, deep learning, fine-tuning, and inference infrastructure, in roughly four to six months part time. Last reviewed: 2026-10-21

The short version

  • An ML engineer trains, adapts, evaluates and serves models. Where an AI engineer treats the model as a service, you treat it as an artifact you own: its data, its weights, its hardware and its latency.
  • You need more foundations than the AI engineer path: Python and NumPy fluency, linear algebra and probability at an applied level, one deep learning framework (PyTorch is the default), and a working knowledge of GPUs.
  • Learn in this order: classical ML (to learn evaluation and overfitting cheaply), deep learning (to learn training), transformers (to understand what everyone ships), fine-tuning (the most common modern task), serving (where the cost lives).
  • Most ML engineering work in 2026 is not pretraining a frontier model. It is fine-tuning open-weight models, building data pipelines, running evaluations, and making inference fast and cheap.
  • The data is usually the hard part. Expect to spend more time on datasets than on architectures.

Who this is for

You can program comfortably in Python. You may be a software engineer, a data scientist, a data engineer, or a student. You want to understand and control models, not just call them. If you would rather build applications on hosted models, the AI engineer path is faster and the job market for it is broader; you can come back here later.

Stage 1: The maths, applied (weeks 1 to 4)

You need less maths than people fear and more than zero. The goal is to read a training loop or a paper's method section and know what each line is doing.

  • Linear algebra: vectors, matrices, matrix multiplication, dot products, and what a shape like (batch, sequence, hidden) means. A neural network layer is a matrix multiply plus a simple function; once that is natural, most architectures are readable.
  • Calculus: what a derivative and a gradient are, and the chain rule. You will not compute gradients by hand (frameworks do that), but you need to know what "the gradient of the loss with respect to the weights" means.
  • Probability and statistics: distributions, expectation, variance, conditional probability, and the logic of a train/validation/test split. A language model outputs a probability distribution over tokens; sampling and decoding is applied probability.

Learn these with code, not just pen and paper: implement a dot product, a softmax and a gradient-descent step in NumPy. Python for AI and Notebooks and Colab cover the tools.

Checkpoint: you can implement linear regression trained by gradient descent in about 30 lines of NumPy, and explain why the loss goes down.

Stage 2: Classical machine learning (weeks 5 to 7)

Start with What is machine learning? for the framing. Then work with scikit-learn on tabular data: linear and logistic regression, decision trees, gradient-boosted trees, k-means. Gradient-boosted trees are still the strongest choice for many tabular business problems, and they train in seconds.

The real lessons of this stage are not the algorithms. They are overfitting, data leakage, choosing a metric, and honest evaluation. These lessons carry straight into deep learning and LLM work, and they are far cheaper to learn on a dataset that fits in memory.

Checkpoint: you have trained a model on a public tabular dataset, beaten a simple baseline on a held-out test set, and can explain one way you could have fooled yourself with leakage.

Stage 3: Deep learning (weeks 8 to 12)

Read Neural networks in plain English, then learn PyTorch: tensors, autograd, nn.Module, a training loop, a data loader. Train a small image classifier and a small text model yourself.

This is where the research summaries start paying off. Read them alongside the code:

Checkpoint: you can write a PyTorch training loop from scratch, train a model on a GPU, and diagnose an overfitting run from its loss curves.

Stage 4: Transformers and language models (weeks 13 to 16)

Read Transformer architecture and the Attention Is All You Need summary, then build one. Andrej Karpathy's "Let's build GPT" video walks through writing a small GPT from scratch in a few hundred lines, and it is the best single exercise on this path.

Then learn what changed after 2017: tokenization, RoPE positional embeddings, grouped-query attention, FlashAttention, mixture of experts, the KV cache, and scaling laws with Chinchilla. The LLaMA and DeepSeek-V3 summaries show how these pieces combine in real models.

Checkpoint: you have trained a small character- or token-level GPT, and can explain what each block in a modern open model's config file controls.

Stage 5: Post-training and fine-tuning (weeks 17 to 20)

This is the most common ML engineering task today. Read Fine-tuning vs RAG first, so you know when not to fine-tune. Then:

Project: Fine-tune with LoRA. Fine-tune a small open-weight model on a narrow task, and compare it against the base model with a prompt, using an eval set you wrote before training. Be prepared for the prompt to win; finding that out is a valid result.

Stage 6: Serving and infrastructure (weeks 21 to 24)

Models cost most when they run, not when they train. Read GPUs for AI, Inference servers, the PagedAttention/vLLM summary, speculative decoding, and the inference economics and AI hardware landscape explainers. For training at scale, ZeRO and Megatron-LM and cost of training.

Projects: Run Llama on a single GPU with an inference server, measure throughput and latency at different batch sizes, then Deploy an ML model behind an API. The LLM serving decision and the AI/ML pipeline pattern frame the production choices, and cloud AI/ML services compares the managed options.

Hardware and budget

You do not need to buy a GPU. Free notebook tiers (Notebooks and Colab) cover Stages 1 to 3, though free GPU access is limited and not guaranteed. For Stages 4 to 6, renting a cloud GPU by the hour is usually cheaper than buying one until you are using it most days. A consumer GPU with 16 to 24 GB of memory is enough to fine-tune small models with QLoRA and to serve 7B-to-30B-class models quantised.

Common mistakes

  • Skipping classical ML. It is where you learn evaluation discipline cheaply. People who skip it make leakage mistakes on expensive GPU runs.
  • Starting from pretraining. Almost no one pretrains a large model. Learn to adapt and serve existing ones first.
  • Fine-tuning to add knowledge. Fine-tuning is good at teaching a format, a style or a narrow skill. It is a poor way to teach facts; use retrieval for that.
  • No baseline. Always compare against the base model with a good prompt. If you cannot beat it, the fine-tune is not worth maintaining.
  • Ignoring data quality. Most fine-tuning failures are data failures: duplicates, label noise, train/test overlap, or training examples that do not look like production inputs.

Next steps

Sources

  • PyTorch tutorials: https://docs.pytorch.org/tutorials/
  • scikit-learn user guide: https://scikit-learn.org/stable/user_guide.html
  • Andrej Karpathy, "Let's build GPT: from scratch, in code, spelled out" (video): https://www.youtube.com/watch?v=kCc8FmEb1nY
  • Hu et al., "LoRA: Low-Rank Adaptation of Large Language Models" (2021): https://arxiv.org/abs/2106.09685
  • Dettmers et al., "QLoRA: Efficient Finetuning of Quantized LLMs" (2023): https://arxiv.org/abs/2305.14314
  • Kwon et al., "Efficient Memory Management for Large Language Model Serving with PagedAttention" (2023): https://arxiv.org/abs/2309.06180
  • Google Colab FAQ (free tier limits are not guaranteed): https://research.google.com/colaboratory/faq.html