747 items · Page 15 of 16
Create a free account to keep your reading organized, join thoughtful discussions, and get more from every chapter.
Free to join · Takes less than a minute
Why
I created this space so readers can learn together, ask questions, and make sense of difficult ideas.
Covers where subword tokenizers fail: number fragmentation, code identifier splitting, multilingual fertility gaps, emoji edge cases.
Covers POS tagging from tag sets to statistical taggers. Topics include Penn Treebank, Universal Dependencies, emission and transition probabilities.
Multiple linear regression estimates how several predictors relate to an outcome. Covers OLS formulas, assumptions, interpretation, diagnostics, and Python.
Conceptual Dependency maps sentences to primitive actions and roles. Covers Roger Schank's theory, semantic equivalence, inference, and question answering.
Explains how NER identifies people, organizations, and locations using BIO sequence labeling. Topics include entity types, boundary detection, nested entities.
SentencePiece trains BPE and unigram tokenizers directly on raw text. Covers whitespace handling, multilingual vocabularies, and model training.
Andrew Viterbi's 1967 dynamic programming algorithm finds the most likely hidden-state sequence, with applications in HMMs, tagging, and speech recognition.
Train custom tokenizers with HuggingFace, covering corpus preparation, vocabulary sizing, algorithm selection, saving, versioning.
Covers multicollinearity in regression analysis with this practical guide. VIF analysis, correlation matrices, coefficient stability testing.
Covers the BIO tagging scheme for named entity recognition, including BIOES variants, span-to-tag conversion, decoding.
The 1954 Georgetown-IBM demonstration marked an important moment in computational linguistics.
Covers Byte Pair Encoding (BPE), the subword tokenization algorithm powering GPT and modern LLMs. Explains how BPE builds a vocabulary.
Explains how GloVe derives word embeddings from co-occurrence ratios, derives the weighted least squares objective.
Covers Ordinary Least Squares (OLS) regression, including mathematical derivations, matrix formulations, step-by-step examples, and Python implementation.
Covers BM25, the major probabilistic ranking algorithm that changed information retrieval. Explains how BM25 solved TF-IDF's limitations.
Explains how FastText extends Word2Vec with character n-grams to handle out-of-vocabulary words, typos.
Evaluate word embeddings using similarity benchmarks, analogy tests, t-SNE visualization, downstream tasks, and bias detection with WEAT.
Traces Richard Montague's major framework for formal natural language semantics. Explains how Montague Grammar introduced compositionality, intensional logic.
Covers Word2Vec training pipeline: subsampling, learning rate scheduling, minibatch training, convergence monitoring.
Simple linear regression models the relationship between one predictor and an outcome. Covers OLS formulas, assumptions, interpretation, and Python.
Explains how hierarchical softmax reduces word embedding training from O(V) to O(log V) using Huffman-coded binary trees, path probabilities.
Michael Lesk's 1983 algorithm resolves word senses by comparing dictionary definitions with surrounding context. Covers the method, code, and limitations.
Explains how word embeddings solve analogies via vector arithmetic, covering the parallelogram model, 3CosAdd and 3CosMul methods, evaluation datasets.
Explains how negative sampling reduces Word2Vec training from O(V) to O(k), covering the binary classification objective, unigram^0.75 sampling distribution.
Covers R-squared, the coefficient of determination. Explains what R-squared means, how to calculate it, interpret its value.
Explains how Gerard Salton's Vector Space Model and TF-IDF weighting changed information retrieval in 1968.
Examines the CBOW Word2Vec architecture covering context averaging, the objective function, gradient derivation, training dynamics.
Explains how the Skip-gram model trains a neural network to predict context words and learns dense word embeddings.
Examines Noam Chomsky's early 1957 work "Syntactic Structures" that changed linguistics, challenged behaviorism.
Explains how Singular Value Decomposition compresses sparse co-occurrence matrices into dense word embeddings through Latent Semantic Analysis.
Generalized linear models combine link functions with probability distributions. Covers logistic and Poisson regression, maximum likelihood, and Python.
Explains how PMI and PPMI transform raw co-occurrence counts into normalized word association scores.
In 2002, IBM researchers introduced BLEU (Bilingual Evaluation Understudy), revolutionizing machine translation evaluation.
Examines raw, log-scaled, boolean, augmented, and L2-normalized term frequency variants, with sparsity analysis and efficient computation using scikit-learn.
Explains how Firth's principle that words are known by their company underpins distributional semantics.
Conditional Random Fields model structured predictions without assuming independent labels. Covers the 2001 paper, feature design, inference, and later uses.
Explains how Inverse Document Frequency (IDF) measures word importance across a corpus by weighting rare, discriminative terms higher than common words.
Covers TF-IDF for text representation, including the core formula, variants, normalization, document similarity, feature extraction, sklearn TfidfVectorizer.
Natural language processing underwent a fundamental shift from symbolic rules to statistical learning.
Explains how perplexity measures language model quality through cross-entropy and information theory.
Covers BM25, the probabilistic ranking algorithm powering modern search engines. Topics include IDF derivation, term saturation, length normalization, BM25L.
Claude Shannon's 1948 work on information theory introduced n-gram models, one of the most core concepts in natural language processing.
Explains how word-word and word-document co-occurrence matrices encode distributional meaning.
Explains how n-gram language models assign probabilities to word sequences using the chain rule, Markov assumption, and MLE.
In 1950, Alan Turing proposed a deceptively simple test for machine intelligence, originally called the Imitation Game.
Covers smoothing techniques that solve the zero probability problem in n-gram models, from simple add-k smoothing through Good-Turing estimation to Kneser-Ney.
Explains how the Bag of Words model transforms text into numerical vectors through word counting, vocabulary construction, and sparse matrix storage.
Joseph Weizenbaum's ELIZA, created in 1966, became the first computer program to hold something resembling a conversation.
Create a free account to keep your reading organized, join thoughtful discussions, and get more from every chapter.
Free to join · Takes less than a minute
Why
I created this space so readers can learn together, ask questions, and make sense of difficult ideas.