7 reports · 2015–2024

Efficiency & deployment

Doing more with less compute: adapters, quantisation, fast attention, sparse experts and deployment trade-offs.

Fig. 02 / Field notesFewer levels, less memory
Quantization maps continuous weight magnitudes to a small set of discrete levels.
Continuous weights are mapped to a small set of discrete values.

Knowledge Distillation: Teacher-Student Compression

Efficiency & deployment · 5 min read · Intermediate

Knowledge distillation compresses big teacher models into small fast students using soft targets — the key to efficient edge AI.

knowledge-distillationcompressionefficiency Read

LoRA: Efficient Fine-Tuning of Large Models

Efficiency & deployment · 4 min read · Advanced

LoRA adapts large models by training small low-rank matrices while freezing the base weights — fine-tuning quality at a fraction of the cost.

lorafine-tuningparameter-efficiency Read

QLoRA: Quantized Fine-Tuning

Efficiency & deployment · 5 min read · Intermediate

QLoRA fine-tunes 65B models on one 48GB GPU by combining 4-bit quantization with LoRA adapters — nearly matching full-precision tuning.

qloraquantizationfine-tuning Read

Mixtral: Sparse Mixture of Experts

Efficiency & deployment · 5 min read · Intermediate

Mixtral routes each token to 2 of 8 experts, matching LLaMA-2 70B quality with 6x faster inference — open sparse MoE that actually works.

mixture-of-expertsmoeefficiency Read

Videos on Efficiency & deployment

Narrated films that touch this topic, with chapters, captions and transcripts.

All videos