712 items · Page 12 of 15
Create a free account to keep your reading organized, join thoughtful discussions, and get more from every chapter.
Free to join · Takes less than a minute
Why
I created this space so readers can learn together, ask questions, and make sense of difficult ideas.
Covers art of communicating with AI agents through effective prompting. Craft clear instructions, use roles and examples.
Explains how weight tying reduces transformer parameters by sharing the input embedding and output projection matrices.
Explains how encoder-only transformers like BERT use bidirectional self-attention for text understanding.
Covers BERT (Bidirectional Encoder Representations from Transformers), including masked language modeling, bidirectional context understanding.
Covers advanced prompting strategies for AI agents including role assignment, few-shot prompting with examples, and iterative refinement.
Explains how GLUs transform feed-forward networks through multiplicative gating. Topics include SwiGLU, GeGLU, and the parameter trade-offs that power LLaMA.
Covers Elastic Net regularization, including mathematical foundations, geometric interpretation.
Compare activation functions in transformer feed-forward networks: ReLU's simplicity and dead neuron problem, GELU's smooth probabilistic gating for BERT.
Explains how XLNet, RoBERTa, and ALBERT refined BERT through permutation language modeling, optimized training procedures, and architectural efficiency.
Covers the fundamentals of writing effective prompts for AI agents. Explains how to be specific, provide context.
Assemble transformer blocks by combining residual connections, normalization, attention, and feed-forward networks.
Explains how layer normalization enables stable transformer training by normalizing.
Model nonlinear relationships with polynomial regression. Covers feature construction, Python implementation, model selection, bias, variance, and overfitting.
Covers preference-based learning, the framework developed by Christiano et al. in 2017.
Language models are the reasoning core of many AI agents. Covers token prediction, instruction following, tool selection, and limits of model-only systems.
Explains how feed-forward networks provide nonlinearity in transformers, with 2-layer architecture, 4x dimension expansion, parameter analysis.
Compare pre-norm and post-norm transformer blocks. Covers layer-normalization placement, gradient flow, training stability, and model depth.
Covers the Transformer architecture, including self-attention mechanisms, multi-head attention, positional encodings.
Covers what you'll build throughout this book: a capable AI agent that remembers conversations, uses tools, plans tasks, and grows smarter with each chapter.
Explains how residual connections solve the vanishing gradient problem in deep networks. Topics include the math behind skip connections, gradient highways.
Covers Ridge regression and L2 regularization, including mathematical foundations, geometric interpretation, bias-variance tradeoff.
Covers RMSNorm, the simpler alternative to LayerNorm used in LLaMA, Mistral, and modern LLMs.
Covers Wikidata, the collaborative multilingual knowledge base launched in 2012. Explains how Wikidata changed structured knowledge representation.
Explains how language models predict text, process tokens, and power AI agents through simple analogies and clear explanations.
Covers sinusoidal position encoding, the deterministic method that gives transformers positional awareness.
Explains why self-attention is blind to word order and what properties positional encodings need.
Compare covariance, correlation, and regression for relationships between variables. Covers interpretation, assumptions, pitfalls, and Python examples.
Covers FastText and subword tokenization, including character n-gram embeddings, handling out-of-vocabulary words, morphological processing.
Explains what distinguishes AI agents from chatbots, exploring perception, reasoning, action, and autonomy.
Explains how RoPE encodes position through vector rotation, making attention scores depend on relative position.
Explains how QKV projections enable transformers to learn flexible attention patterns through specialized query, key, and value representations.
Covers residual connections, the architectural innovation that solved the vanishing gradient problem in deep networks.
Compare transformer position encoding methods including sinusoidal, learned embeddings, RoPE, and ALiBi.
Explains how relative position encoding improves transformer generalization by encoding token distances rather than absolute positions.
Covers Google's transition to neural machine translation in 2016. Explains how GNMT replaced statistical phrase-based methods with end-to-end neural networks.
How GPT and BERT encode position through learnable parameters. Topics include embedding tables, position similarity, interpolation techniques.
Explains how ALiBi encodes position through linear attention biases instead of embeddings. Topics include head-specific slopes, extrapolation properties.
Covers statistical modeling fundamentals, including measuring model fit with R-squared and RMSE.
Covers sequence-to-sequence neural machine translation, the 2014 advance that changed translation from statistical pipelines to end-to-end neural models.
Explains how multi-head attention runs multiple attention operations in parallel, enabling transformers to capture diverse relationships like syntax, semantics.
Self-attention has O(n²d) compute and quadratic memory growth. Covers when sparse, linear, and other efficient attention variants reduce those costs.
Covers GloVe (Global Vectors) and the Adam optimizer, two early 2014 developments that changed neural language processing.
Covers scaled dot-product attention: how queries and keys compute similarity, why dividing by sqrt(d_k) prevents softmax saturation.
Covers attention masking techniques including padding masks, causal masks, and sparse patterns.
The application of deep neural networks to speech recognition in 2012, led by Geoffrey Hinton and his colleagues, marked a major advance.
Self-attention relates every token to every other token in a sequence. Covers queries, keys, values, contextual embeddings, masking, and computational cost.
Explains how beam search balances quality and compute in sequence generation, covering greedy decoding limits, length normalization, diverse beam search.
Covers the Gauss-Markov assumptions that underpin linear regression. Topics include the five key assumptions, how to test them, consequences of violations.
Create a free account to keep your reading organized, join thoughtful discussions, and get more from every chapter.
Free to join · Takes less than a minute
Why
I created this space so readers can learn together, ask questions, and make sense of difficult ideas.