Deep RL from Human Preferences
Christiano et al. trained agents from human preferences instead of hand-written rewards — the 2017 result that made RLHF, and ChatGPT, possible.
4 reports · 2017–2023
Turning raw predictors into assistants: human feedback, instruction tuning, preference optimisation and self-critique.
Christiano et al. trained agents from human preferences instead of hand-written rewards — the 2017 result that made RLHF, and ChatGPT, possible.
Constitutional AI trains harmless assistants from a written constitution via AI self-critique, cutting human labels while keeping helpfulness.
An AI-generated review of InstructGPT, the paper that introduced instruction fine-tuning with human feedback (RLHF) and defined how aligned LLMs are built.
Direct Preference Optimization aligns language models from preference pairs with one stable loss — no reward model, no reinforcement learning loop.
Narrated films that touch this topic, with chapters, captions and transcripts.
8:53Essay film · 10 chapters
Five common arguments against AI, taken seriously one at a time and tested against history and published data — plus what the critics get right.