Advanced level (optional): specialisations

Unit 1 · GPUs and CUDA

4 lessons · ≈ 13 h of study

1. GPU architecture
2. The CUDA programming model
3. Roofline and kernel optimisation
4. The memory hierarchy in depth — caches, loop order and tile shape

See the unit: GPUs and CUDA

Unit 2 · LLM performance

5 lessons · ≈ 16 h of study

1. Training and inference memory
2. Quantization, pruning and distillation
3. Batching, throughput and latency
4. Tensor and pipeline parallelism
5. Kernel fusion and FlashAttention — the online softmax

See the unit: LLM performance

Unit 3 · Distributed AI

7 lessons · ≈ 21 h of study

1. Data parallelism and all-reduce
2. Model and optimizer sharding (ZeRO and FSDP)
3. Fault tolerance, checkpoints and resumption
4. Tensor parallelism in depth — the complete Transformer block
5. Pipeline schedules — GPipe, 1F1B and interleaved
6. Mixed precision, loss scaling and overlapping communication
7. Distributed inference — serving a model that doesn't fit on one GPU

See the unit: Distributed AI

Unit 4 · MLOps and infrastructure

7 lessons · ≈ 22 h of study

1. Serving models in production
2. Observability, percentiles and drift
3. Reproducibility, tracking and CI/CD for models
4. Cost, caching, secrets and privacy
5. Paged KV cache — virtual memory for serving LLMs
6. Quantizing for serving — what speeds up, what fits and what breaks
7. Progressive delivery — shadow, canary and rollback

See the unit: MLOps and infrastructure

Unit 5 · Research Engineering

5 lessons · ≈ 15 h of study

1. Reading papers like a researcher
2. Experimental design, seeds and uncertainty
3. Ablations, benchmark hygiene and reproduction
4. Technical writing, honest figures and uncertainty
5. Reproducing a paper — from the PDF to a number you can defend

See the unit: Research Engineering

Unit 6 · Multimodal AI

5 lessons · ≈ 18 h of study

1. Contrastive learning and CLIP — images and texts on the same map
2. Audio and speech — from the microphone to the mel spectrogram
3. Vision-language models and multimodal agents
4. Generative models — autoencoders, VAEs and GANs
5. Image generation — diffusion models

See the unit: Multimodal AI

Unit 7 · The AI frontier

A living area that evolves with the field. Reasoning and inference-time compute, long context, Mixture of Experts, synthetic data and alignment with preferences. Model compression, agents and multimodality have their own units ("LLM performance", "AI agents" and "Multimodal AI").

8 lessons · ≈ 26 h of study

1. Reasoning and inference-time compute
2. Long context and efficient attention
3. Mixture of Experts
4. Alignment and synthetic data
5. The reward model — from human rankings to a number
6. RLHF with PPO — reward with KL, advantages and clipping
7. DPO — the full derivation and why it works
8. Evaluating alignment — reward hacking, over-optimization and biased judges

See the unit: The AI frontier

Unit 8 · Final project — AI systems engineering

A complete AI system, from the interface to deployment, defended technically.

2 lessons · ≈ 8 h of study

1. Designing an AI system end to end
2. Technical defence — justifying every decision with numbers

See the unit: Final project — AI systems engineering