Artificial intelligence
Query, Key, Value: The Foundation of Transformer Attention
Explains how QKV projections enable transformers to learn flexible attention patterns through specialized query, key, and value representations.

Michael Brenndoerfer
I work across AI, technology, and private equity.
Here, I share the ideas behind the work.
Books
Machine learning, language AI, and quantitative finance, explained from first principles.

The math, intuition, and Python behind the algorithms.
Free to read online
From classical NLP to transformers and modern language models.
In progress · Read available chapters

Community
Create a free account to keep your reading organized and take part in discussions.
Create your free account
By continuing, you agree to our Terms and Privacy Policy.
Writing
Artificial intelligence
Explains how QKV projections enable transformers to learn flexible attention patterns through specialized query, key, and value representations.
Finance & markets
Market crashes erase quoted value rather than moving a fixed pool of cash. Covers forced selling, liquidity, wealth effects, price discovery, and recovery.
Language AI
Examines neural text-to-speech systems from acoustic modeling to vocoding. Topics include Tacotron, FastSpeech.
Finance & markets
Covers quadratic programming (QP) for portfolio optimization, including the mean-variance framework, efficient frontier construction.
Machine learning
Covers XGBoost (eXtreme Gradient Boosting), including second-order Taylor expansion, regularization techniques, split gain optimization, ranking loss functions.
Machine learning
Covers Ordinary Least Squares (OLS) regression, including mathematical derivations, matrix formulations, step-by-step examples, and Python implementation.