712 items · Page 8 of 15
Create a free account to keep your reading organized, join thoughtful discussions, and get more from every chapter.
Free to join · Takes less than a minute
Why
I created this space so readers can learn together, ask questions, and make sense of difficult ideas.
Explains why larger language models sometimes perform worse on specific tasks. Topics include distractor tasks, sycophancy, and U-shaped scaling patterns.
Examines whether LLM emergent capabilities are genuine phase transitions or measurement artifacts.
Explains how chain-of-thought reasoning emerges in large language models. Topics include CoT prompting techniques, scaling behavior.
Explains how in-context learning emerges in large language models. Topics include scale thresholds, ICL vs fine-tuning, induction heads, and meta-learning.
Explains how LLMs suddenly acquire capabilities through emergence. Topics include phase transitions, scaling behaviors, and the ongoing metric artifact debate.
Transform scaling laws into predictive tools for AI development. Topics include loss extrapolation, capability forecasting.
Bahdanau attention replaced fixed context vectors with dynamic source alignment, improving neural machine translation and shaping later transformer models.
Chinchilla-optimal training can be inefficient for deployment. Covers deliberate over-training, inference workloads, and total cost modeling for LLM systems.
Examines data-constrained scaling for LLMs: repetition penalties, modified Chinchilla laws, synthetic data strategies, and optimal compute allocation.
Covers derivatives, gradients, and optimization techniques essential for quantitative finance. Topics include Greeks, portfolio optimization.
Explains how DeepMind's Chinchilla scaling laws changed LLM training by proving models should use 20 tokens per parameter for compute-optimal performance.
Explains how power laws govern neural network scaling. Topics include log-log analysis, fitting techniques, and how to predict model performance at any scale.
Covers BART's denoising pre-training approach including text infilling, token masking, sentence permutation, and how corruption schemes enable generation.
Explains how T5 reformulates all NLP tasks as text-to-text problems. Topics include task prefixes, classification, NER.
Covers compute-optimal LLM training using Chinchilla scaling laws. Topics include the 20:1 token ratio, practical allocation formulas.
Covers hybrid retrieval systems introduced in 2024. Explains how hybrid systems combine sparse retrieval for fast candidate generation.
Covers structured outputs introduced in language models during 2024. Explains how structured outputs enable reliable data extraction.
Covers multimodal integration in 2024, the advance that enabled AI systems to directly process and understand text, images, audio.
Covers advanced parameter-efficient fine-tuning methods introduced in 2024, including AdaLoRA, DoRA, VeRA, and other innovations.
Covers continuous post-training, including parameter-efficient fine-tuning with LoRA, catastrophic forgetting prevention, incremental model updates.
Covers DBSCAN (Density-Based Spatial Clustering of Applications with Noise), the algorithm.
Covers GPT-4o, including unified multimodal architecture, real-time processing, unified tokenization, advanced attention mechanisms, memory mechanisms.
Covers quadratic programming (QP) for portfolio optimization, including the mean-variance framework, efficient frontier construction.
DeepSeek R1 uses reinforcement learning and distilled reasoning models for mathematical and coding tasks. Covers its training design, results, and limitations.
Covers agentic AI systems introduced in 2024. Explains how AI systems evolved from reactive tools to autonomous agents capable of planning.
Covers Vehicle Routing Problem with Time Windows (VRPTW), including mathematical formulation, constraint programming.
AI co-scientist systems generate hypotheses, plan experiments, and synthesize evidence. Covers autonomous research workflows, evaluation, and human oversight.
Covers minimum cost flow optimization for slotting problems, including network flow theory, mathematical formulation.
Covers V-JEPA 2, including vision-based world modeling, joint embedding predictive architecture, visual prediction, embodied AI.
Transform raw text into structured data through tokenization, normalization, and cleaning techniques.
Covers TF-IDF and Bag of Words, including term frequency, inverse document frequency, vectorization, and text classification.
Covers word embeddings covering Word2Vec skip-gram, GloVe matrix factorization, negative sampling.
Examines Mistral AI's Mixtral models and how they demonstrated that sparse mixture-of-experts architectures could be production-ready.
Covers Mixed Integer Linear Programming (MILP) for factory optimization, covering mathematical foundations, constraint modeling, branch-and-bound algorithms.
Covers specialized large language models for low-resource languages, including synthetic data generation, cross-lingual transfer learning.
Scale AI agents from single users to thousands while maintaining performance and controlling costs. Topics include horizontal scaling, load balancing.
Covers CP-SAT rostering using Google OR-Tools to solve complex workforce scheduling problems with binary decision variables, coverage constraints.
Covers Constitutional AI, including principle-based alignment, self-critique training, reinforcement learning from AI feedback (RLAIF), scalability advantages.
Reduce AI agent API costs without sacrificing capability. Topics include model selection, caching, batching, prompt optimization.
Examines multimodal large language models that integrated vision and language capabilities, enabling AI systems to process images and text together.
Covers practical techniques to make AI agents respond faster, including model selection strategies, response caching, streaming, parallel execution.
Covers NHITS (Neural Hierarchical Interpolation for Time Series), a deep learning architecture for multi-scale time series forecasting.
Covers the 2023 open LLM wave, including MPT, Falcon, Mistral, and other open models. Explains how these models created a competitive ecosystem.
Maintain and update AI agents safely, manage costs, respond to user feedback, and keep your system healthy over months and years of operation.
Covers N-BEATS, an interpretable deep learning architecture for time series forecasting.
LLaMA showed that carefully trained smaller models could match larger systems. Covers model sizes, data choices, efficiency, and open research access.
Monitor your deployed AI agent's health, handle errors gracefully, and build reliability through health checks, metrics tracking, error handling.
Covers GPT-4, including multimodal capabilities, improved reasoning abilities, enhanced safety and alignment, human-level performance on standardized tests.
Create a free account to keep your reading organized, join thoughtful discussions, and get more from every chapter.
Free to join · Takes less than a minute
Why
I created this space so readers can learn together, ask questions, and make sense of difficult ideas.