course bible
A first-year student should stay oriented; a graduate student should see the real machinery and the places where the field is still unsettled.
The course is designed as an interactive book: intuition first, then the mathematical object, then a lab or demo, then the failure mode. A reader should be able to move from first principles to modern AI systems without losing the thread.
The course treats history as a debugging aid. Each breakthrough changed what researchers could optimize, measure, or scale.
Alan Turing frames machine intelligence as a behavioral question: can a machine carry on an imitation game well enough to make the boundary hard to police?
Frank Rosenblatt demonstrates the Mark I Perceptron, turning weighted votes from examples into a public symbol of learning machines.
Arthur Samuel uses the phrase machine learning while building a checkers program that improves from play.
Rumelhart, Hinton, and Williams make backpropagation central to neural-network training, showing how one loss can teach hidden layers.
LeNet shows convolutional networks working at production scale for handwritten digit recognition.
AlexNet wins ImageNet by a large margin and makes GPU-trained deep learning impossible to ignore.
Attention Is All You Need introduces the transformer architecture that later becomes the workhorse for language, code, vision, and multimodal systems.
Scaling-law work turns model size, data size, and compute into an empirical design surface rather than guesswork.
Diffusion image models and conversational language models make generative AI visible to the public, raising the stakes for evaluation and alignment.
What machine learning is, what it optimizes, and why prediction is a broader idea than forecasting.
You can describe any supervised learning system using data, model, loss, optimizer, and generalization.
The phase establishes empirical risk minimization as the recurring mathematical frame.
How to know whether a model learned a pattern, memorized a sample, leaked data, or merely optimized the wrong score.
You can design train/validation/test splits, choose metrics, and explain why held-out performance matters.
The phase introduces distribution shift, bias-variance, calibration, and metric design as scientific controls.
Boundaries, weighted votes, gradients, and the first models simple enough to understand completely.
You can derive how a line becomes a classifier and how a local update changes the decision boundary.
The phase ties geometry to convex objectives, stochastic gradients, and parameter identifiability.
Layers, nonlinear representations, backpropagation, initialization, optimizers, and regularization.
You can explain how a scalar loss teaches many layers and why training stability is an engineering problem.
The phase treats autodiff, residual pathways, normalization, and implicit regularization as core machinery.
Architectures that exploit structure: images have locality, sequences have order, and attention creates content-addressed retrieval.
You can recognize the inductive bias behind CNNs, RNNs, and attention instead of memorizing architectures.
The phase connects weight sharing, receptive fields, hidden state, and quadratic attention cost.
Transformers, scaling laws, generation, multimodal embeddings, retrieval, and tool-using systems.
You can trace how next-token training becomes a general interface for language, images, retrieval, and tools.
The phase separates pretraining, post-training, retrieval augmentation, and system evaluation.
Learning from reward, aligning behavior with human intent, evaluating deployed systems, and thinking about frontier risk.
You can reason about policies, preference models, eval suites, reward hacking, and agent reliability.
The phase treats evaluation, oversight, interpretability, and governance as technical design constraints.