712 items · Page 11 of 15
Create a free account to keep your reading organized, join thoughtful discussions, and get more from every chapter.
Free to join · Takes less than a minute
Why
I created this space so readers can learn together, ask questions, and make sense of difficult ideas.
Explains how causal language modeling trains AI to predict the next token. Topics include autoregressive factorization, cross-entropy loss, causal masking.
Examines how Mixture of Experts (MoE) architectures changed large language model scaling in 2024.
Build a complete AI agent memory system combining conversation history and persistent knowledge storage.
Explains how Transformer-XL uses segment-level recurrence to extend effective context length.
Explains how Position Interpolation extends transformer context windows by scaling position indices to stay within training distributions.
Covers boosted trees and gradient boosting, covering ensemble learning, loss functions, sequential error correction, and scikit-learn implementation.
Covers OpenAI's DALL·E 2, the major text-to-image generation model that combined CLIP-guided diffusion with high-quality image synthesis.
Explains how AI agents store and retrieve information across sessions using vector databases, embeddings, and semantic search.
Examines the fundamental barriers limiting transformer context windows, including quadratic memory scaling, position encoding extrapolation failures.
Explains why the first tokens in transformer sequences absorb excess attention weight, how this causes streaming inference failures.
Codex adapted GPT-3 for code completion and generation. Covers its training data, natural-language prompts, benchmark results, and role in developer tools.
Give AI agents the ability to remember recent conversations, handle follow-up questions, and manage conversation history across multiple interactions.
Covers Random Forest covering ensemble learning, bootstrap sampling, random feature selection, bias-variance tradeoff, and implementation in scikit-learn.
NTK-aware scaling extends transformer context windows by adjusting rotary frequencies. Covers position preservation, longer sequences, and tradeoffs.
Covers OpenAI's DALL·E, the early text-to-image generation model that extended transformer architectures to multimodal tasks.
Build a working calculator tool for your AI agent from scratch. Topics include the complete workflow from Python function to tool integration.
Covers FlashAttention's tiled computation and online softmax algorithms. Topics include GPU memory hierarchy, CUDA kernel basics.
Explains how FlashAttention achieves 2-4x speedups by restructuring attention computation. Topics include GPU memory hierarchy, tiling for SRAM.
Covers CART (Classification and Regression Trees), including mathematical foundations, Gini impurity, variance reduction.
GPT-3 showed that scaling autoregressive transformers could enable in-context and few-shot learning. Covers its architecture, data, prompting, and limitations.
Call language models from Python code, including GPT-5, Claude Sonnet 4.5, and Gemini 2.5. Covers API integration, error handling.
Explains how YaRN extends LLM context length through wavelength-based frequency interpolation and attention temperature correction.
Explains how linear attention achieves O(nd²) complexity by replacing softmax with kernel functions.
Covers Google's T5 (Text-to-Text Transfer Transformer) introduced in 2019. Explains how the text-to-text framework unified diverse NLP tasks.
Design effective tool interfaces for AI agents, from basic function definitions to multi-tool orchestration.
Explains how sliding window attention reduces transformer complexity from quadratic to linear.
Covers logistic regression covering mathematical foundations, the logistic function, optimization algorithms, and practical implementation.
Explains how Longformer combines sliding window and global attention to process documents of 4,096+ tokens with O(n) complexity instead of O(n²).
Covers GLUE and SuperGLUE benchmarks introduced in 2018. Explains how these standardized evaluation frameworks changed language AI research.
Explains why AI agents need external tools to overcome limitations like outdated knowledge, imprecise calculations, and inability to take real-world actions.
Implement sparse attention patterns including local windows, strided attention, and block-sparse methods.
BigBird combines sliding-window, global, and random attention to reach linear complexity. Covers its guarantees and use with long documents.
Covers Poisson regression for count data analysis. Topics include mathematical foundations, maximum likelihood estimation, rate ratio interpretation.
Transformer-XL extends context with segment recurrence and relative positions. Covers its architecture, efficiency, training method, and influence.
Use chain-of-thought prompting to get AI agents to reason through problems step by step, improving accuracy and transparency for complex questions.
Global tokens create communication hubs in sparse attention, reducing path length and helping efficient transformers exchange information across long documents.
Explains why self-attention has O(n²) complexity, how memory and compute scale quadratically with sequence length.
Covers BERT's application to information retrieval in 2019. Explains how transformer architectures changed search and ranking systems.
Guide AI agents to verify and refine their reasoning through self-checking techniques. Topics include practical methods for catching errors, improving accuracy.
Covers encoder-decoder transformer architecture that powers T5 and machine translation. Topics include cross-attention mechanism.
Spline regression models nonlinear relationships with piecewise polynomials. Covers B-splines, knot selection, natural cubic splines, and Python.
Covers decoder-only transformers powering GPT, Llama, and modern LLMs. Topics include causal masking, autoregressive generation, KV caching.
Covers ELMo and ULMFiT, the advance methods that established transfer learning for NLP in 2018.
Teach AI agents to think through problems step by step using chain-of-thought reasoning.
Design transformer architectures by understanding the key hyperparameters: model depth, width, attention heads, and FFN dimensions.
Covers cross-attention, the mechanism that bridges encoder and decoder in sequence-to-sequence transformers.
Covers multinomial logistic regression covering mathematical foundations, softmax function, coefficient estimation.
Covers OpenAI's GPT-1 and GPT-2 models. Explains how autoregressive pretraining with transformers enabled transfer learning across NLP tasks.
Create a free account to keep your reading organized, join thoughtful discussions, and get more from every chapter.
Free to join · Takes less than a minute
Why
I created this space so readers can learn together, ask questions, and make sense of difficult ideas.