Research Engineering

Topics covered

• Papers
• Reproduction
• Experiments
• Ablations
• Benchmarks
• Hypotheses
• Technical writing
• Methodology

Lessons in this unit

1. Reading papers like a researcher 65 min
   The three-pass method and how to separate claims, hypotheses and evidence to decide what to believe in a paper.
2. Experimental design, seeds and uncertainty 85 min
   Why a single number lies, how to measure variability with seeds and the bootstrap, how to compare two models in a paired way and how not to fool yourself by trying many variants.
3. Ablations, benchmark hygiene and reproduction 85 min
   How to isolate which part of a method produces the improvement, why components interact, how to detect leakage and contamination in a benchmark and how to reproduce a published result.
4. Technical writing, honest figures and uncertainty 65 min
   How to write a technical report that can be read in two minutes and defended for an hour, how to draw figures that don't exaggerate and how to report uncertainty with the right number of digits.
5. Reproducing a paper — from the PDF to a number you can defend 60 min
   A working method for reproducing results — choosing the claim, extracting the specification, logging deviations, verifying equations with tests — and two NumPy mini-reproductions of classic claims from Attention Is All You Need and from Adam.

Prerequisites

Before this unit it helps to have done:

• MLOps and infrastructure (Course 5 · Unit 4)