Data, Analytics & AILanguage AI HandbookMachine Learning
The Position Problem
Jun 3, 2025·47 min read
Explains why self-attention is blind to word order and what properties positional encodings need.
Open notebook
Data, Analytics & AI
Articles about data, analytics, and AI, including machine learning, data visualization, and AI applications.
702 items · Page 12 of 15
Community
Learn with the community
Create a free account to keep your reading organized, join thoughtful discussions, and get more from every chapter.
- Keep your place across books and articles
- Ask questions and take part in discussions
- Read with fewer interruptions
Free to join · Takes less than a minute
Why
I created this space so readers can learn together, ask questions, and make sense of difficult ideas.
From readers1 / 12
Data, Analytics & AIMachine LearningMachine Learning from Scratch
Variable Relationships: Covariance, Correlation
Jun 3, 2025·26 min read
Compare covariance, correlation, and regression for relationships between variables. Covers interpretation, assumptions, pitfalls, and Python examples.
Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI
Subword Tokenization and FastText
Jun 3, 2025·14 min read
Covers FastText and subword tokenization, including character n-gram embeddings, handling out-of-vocabulary words, morphological processing.
Open notebook
AI Agent HandbookMachine LearningData, Analytics & AI
What Is an AI Agent? Components and Autonomy
Jun 3, 2025·10 min read
Explains what distinguishes AI agents from chatbots, exploring perception, reasoning, action, and autonomy.
Open notebook
Data, Analytics & AIMachine LearningLanguage AI Handbook
Rotary Position Embedding (RoPE)
Jun 2, 2025·54 min read
Explains how RoPE encodes position through vector rotation, making attention scores depend on relative position.
Open notebook
Data, Analytics & AIMachine LearningLanguage AI Handbook
Query, Key, Value: The Foundation of Transformer Attention
Jun 1, 2025·40 min read
Explains how QKV projections enable transformers to learn flexible attention patterns through specialized query, key, and value representations.
Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI
Residual Connections
Jun 1, 2025·15 min read
Covers residual connections, the architectural innovation that solved the vanishing gradient problem in deep networks.
Open notebook
Data, Analytics & AIMachine LearningLanguage AI Handbook
Position Encoding Comparison: Sinusoidal, Learned, RoPE
May 31, 2025·38 min read
Compare transformer position encoding methods including sinusoidal, learned embeddings, RoPE, and ALiBi.
Open notebook
Data, Analytics & AIMachine Learning from Scratch
Data Quality & Outliers: Measurement Error, Missing Data
May 31, 2025·34 min read
Assess data quality through measurement error, systematic bias, missing-data mechanisms, and outlier detection. Covers diagnosis, treatment, and validation.
Open notebook
Data, Analytics & AILanguage AI HandbookMachine Learning
Relative Position Encoding
May 30, 2025·49 min read
Explains how relative position encoding improves transformer generalization by encoding token distances rather than absolute positions.
Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI
Google NMT: End-to-End Neural Machine Translation
May 30, 2025·14 min read
Covers Google's transition to neural machine translation in 2016. Explains how GNMT replaced statistical phrase-based methods with end-to-end neural networks.
Open notebook
Data, Analytics & AIMachine LearningLanguage AI Handbook
Learned Position Embeddings
May 29, 2025·48 min read
How GPT and BERT encode position through learnable parameters. Topics include embedding tables, position similarity, interpolation techniques.
Open notebook
Data, Analytics & AIMachine LearningLanguage AI Handbook
ALiBi: Attention with Linear Biases for Position Encoding
May 28, 2025·52 min read
Explains how ALiBi encodes position through linear attention biases instead of embeddings. Topics include head-specific slopes, extrapolation properties.
Open notebook
Data, Analytics & AIMachine LearningMachine Learning from Scratch
Statistical Modeling: Overfitting, Bias and Variance
May 28, 2025·19 min read
Covers statistical modeling fundamentals, including measuring model fit with R-squared and RMSE.
Open notebook
History of Language AIMachine LearningData, Analytics & AI
Seq2Seq NMT: End-to-End Learning Revolution
May 28, 2025·30 min read
Covers sequence-to-sequence neural machine translation, the 2014 advance that changed translation from statistical pipelines to end-to-end neural models.
Open notebook
Data, Analytics & AISoftware EngineeringLLM and GenAI
What are AI Agents, Really?
May 27, 2025·10 min read
AI agents combine a language model with tools, memory, and control logic. This article separates agents from workflows and multi-agent systems.
Read article
Data, Analytics & AIMachine LearningLanguage AI Handbook
Multi-Head Attention
May 27, 2025·44 min read
Explains how multi-head attention runs multiple attention operations in parallel, enabling transformers to capture diverse relationships like syntax, semantics.
Open notebook
Data, Analytics & AILanguage AI HandbookMachine Learning
Attention Complexity: Quadratic Scaling, Memory Limits
May 26, 2025·62 min read
Self-attention has O(n²d) compute and quadratic memory growth. Covers when sparse, linear, and other efficient attention variants reduce those costs.
Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI
GloVe and Adam Optimizer: Global Word Embeddings
May 26, 2025·25 min read
Covers GloVe (Global Vectors) and the Adam optimizer, two early 2014 developments that changed neural language processing.
Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningLanguage AI Handbook
Scaled Dot-Product Attention: The Core Transformer Mechanism
May 25, 2025·48 min read
Covers scaled dot-product attention: how queries and keys compute similarity, why dividing by sqrt(d_k) prevents softmax saturation.
Open notebook
Data, Analytics & AIdata-scienceMachine Learning from Scratch
Data Visualization: Histograms, Box Plots and Scatter Plots
May 25, 2025·17 min read
Covers core data visualization techniques including histograms, box plots, and scatter plots. Topics include distributions, identify outliers.
Open notebook
Data, Analytics & AIMachine LearningLanguage AI Handbook
Attention Masking: Controlling Information Flow
May 24, 2025·52 min read
Covers attention masking techniques including padding masks, causal masks, and sparse patterns.
Open notebook
Data, Analytics & AIMachine LearningLLM and GenAIHistory of Language AI
Deep Learning for Speech Recognition: The 2012 DNN Results
May 24, 2025·13 min read
The application of deep neural networks to speech recognition in 2012, led by Geoffrey Hinton and his colleagues, marked a major advance.
Open notebook
Data, Analytics & AIMachine LearningLanguage AI Handbook
Self-Attention Concept
May 23, 2025·50 min read
Self-attention relates every token to every other token in a sequence. Covers queries, keys, values, contextual embeddings, masking, and computational cost.
Open notebook
Data, Analytics & AISoftware EngineeringLLM and GenAI
Model Context Protocol (MCP) for LLM Tool Integration
May 22, 2025·6 min read
Model Context Protocol (MCP) gives LLM applications a standard way to list tools, call them, and exchange structured results across services.
Read article
Data, Analytics & AISoftware EngineeringMachine LearningLanguage AI Handbook
Beam Search: Decoding and Sequence Generation
May 22, 2025·48 min read
Explains how beam search balances quality and compute in sequence generation, covering greedy decoding limits, length normalization, diverse beam search.
Open notebook
Data, Analytics & AIMachine LearningMachine Learning from Scratch
Gauss-Markov Assumptions for OLS Regression
May 22, 2025·17 min read
Covers the Gauss-Markov assumptions that underpin linear regression. Topics include the five key assumptions, how to test them, consequences of violations.
Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningLanguage AI Handbook
Teacher Forcing: Training Seq2Seq with Ground Truth Context
May 21, 2025·45 min read
Teacher forcing trains sequence-to-sequence models with ground-truth context. Covers exposure bias, scheduled sampling, and alternatives based on REINFORCE.
Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningLanguage AI Handbook
Bidirectional RNNs: Full-Sequence Context for NLP
May 20, 2025·50 min read
Explains how bidirectional LSTMs and GRUs provide full-sequence context by processing text forward and backward, enabling strong sequence labeling.
Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI
Neural Search: Semantic Search with Deep Learning
May 20, 2025·21 min read
Covers neural information retrieval, the advance approach that learned semantic representations for queries and documents.
Open notebook
Data, Analytics & AIMachine LearningLanguage AI Handbook
Bahdanau Attention: Dynamic Context for NMT
May 19, 2025·51 min read
Explains how Bahdanau attention solves the encoder-decoder bottleneck with dynamic context vectors, softmax alignment.
Open notebook
Data, Analytics & AIMachine LearningMachine Learning from Scratch
Normalization: Feature Scaling with Min-Max Implementation
May 19, 2025·14 min read
Covers normalization in machine learning, covering min-max scaling, proper train-test split implementation, when to use normalization vs standardization.
Open notebook
Data, Analytics & AISoftware EngineeringLLM and GenAI
Why Temperature=0 Doesn't Guarantee Determinism in LLMs
May 18, 2025·12 min read
Temperature zero does not make an LLM deterministic. GPU kernels, batching, floating-point arithmetic, and serving choices can still change its output.
Read article
Data, Analytics & AISoftware EngineeringMachine LearningLanguage AI Handbook
Luong Attention: Dot Product, General & Local Mechanisms
May 18, 2025·43 min read
Examines Luong attention variants including dot product, general, and concat scoring. Topics include global vs local attention, input-feeding.
Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI
Layer Normalization for Sequence Models
May 18, 2025·13 min read
Covers layer normalization, the normalization technique that computes statistics across features for each example.
Open notebook
Data, Analytics & AILanguage AI HandbookMachine Learningdeep-learningnatural-language-processing
Copy Mechanism: Pointer Networks for Neural Text Generation
May 17, 2025·60 min read
Explains how copy mechanisms enable seq2seq models to handle out-of-vocabulary words.
Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningLanguage AI Handbook
Attention Mechanism Intuition: Soft Lookup, Weights
May 16, 2025·56 min read
Explains how attention mechanisms solve the encoder-decoder bottleneck through soft lookup, alignment scores, and dynamic context vectors built step by step.
Open notebook
Data, Analytics & AIMachine Learning from Scratch
Statistical Sampling: Populations, Samples and Bias
May 16, 2025·16 min read
Covers sampling theory and methods in data science, covering simple random sampling, stratified sampling, cluster sampling, sampling error.
Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI
Word2Vec: Neural Word Embeddings and Vector Semantics
May 16, 2025·22 min read
Covers word2vec, the advance method for learning dense vector representations of words.
Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningLanguage AI Handbook
Encoder-Decoder Framework
May 15, 2025·56 min read
Covers the encoder-decoder framework for sequence-to-sequence learning. Topics include context vectors, teacher forcing, BLEU evaluation.
Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningLanguage AI Handbook
GRU Architecture: Streamlined Gating for Sequence Modeling
May 14, 2025·48 min read
Gated Recurrent Units use reset and update gates to manage sequence memory. Covers GRU equations, implementation, gradient flow, and comparison with LSTMs.
Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI
SQuAD: Reading Comprehension Benchmark
May 14, 2025·16 min read
Covers SQuAD (Stanford Question Answering Dataset), the benchmark that established reading comprehension as a flagship NLP task.
Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningLanguage AI Handbook
Stacked RNNs and Hierarchical Modeling
May 13, 2025·41 min read
Stacked RNNs build hierarchical representations across recurrent layers. Covers depth-width tradeoffs, variational dropout, ELMo, and practical depth limits.
Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningLanguage AI Handbook
Semantic Parsing and Meaning Representations
May 13, 2025·61 min read
Learn how semantic parsing turns language into logical forms, AMR graphs, and executable SQL, with schema linking, constrained generation, and evaluation.
Open notebook
Data, Analytics & AIMachine Learning from ScratchMachine Learning
Probability Distributions: Normal, Binomial, Poisson
May 13, 2025·18 min read
Covers probability distributions for data science, including normal, t-distribution, binomial, Poisson, exponential, and log-normal distributions.
Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningLanguage AI Handbook
LSTM Gradient Flow: Constant Error Carousel Explained
May 12, 2025·52 min read
How LSTMs solve the vanishing gradient problem through the cell state gradient highway, forget gate modulation, and the residual connection analogy.
Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningLanguage AI Handbook
Entity Linking and Knowledge Graph Construction
May 12, 2025·70 min read
Link text mentions to knowledge-base entities, rank candidates with context and document coherence, and build a graph whose assertions retain source evidence.
Open notebook
Data, Analytics & AIMachine LearningLLM and GenAIHistory of Language AI
WaveNet: Neural Audio Generation Revolution
May 12, 2025·16 min read
DeepMind's WaveNet changed text-to-speech synthesis in 2016 by generating raw audio waveforms directly using neural networks.
Open notebook
Explore Other Categories
Community
Learn with the community
Create a free account to keep your reading organized, join thoughtful discussions, and get more from every chapter.
- Keep your place across books and articles
- Ask questions and take part in discussions
- Read with fewer interruptions
Free to join · Takes less than a minute
Why
I created this space so readers can learn together, ask questions, and make sense of difficult ideas.
From readers1 / 12