712 items · Page 14 of 15
Create a free account to keep your reading organized, join thoughtful discussions, and get more from every chapter.
Free to join · Takes less than a minute
Why
I created this space so readers can learn together, ask questions, and make sense of difficult ideas.
Covers Adam optimizer: first and second moment estimates, bias correction, adaptive learning rates, hyperparameter tuning.
In 1998, Charles Fillmore's FrameNet project at ICSI Berkeley released the first large-scale computational resource based on frame semantics.
Momentum smooths gradient updates with a velocity term. Covers Polyak momentum, Nesterov acceleration, damped oscillations, and optimization tradeoffs.
Explains how gradient clipping prevents training instability by capping gradient magnitudes. Topics include clip by value vs clip by norm strategies.
Covers the Sum of Squared Errors (SSE) metric in regression analysis. Topics include the mathematical foundation, visualization techniques.
Examines John Searle's influential 1980 thought experiment challenging strong AI.
Covers every major activation function from sigmoid to GELU. Topics include saturation, dying ReLU, gradient flow analysis.
Covers AdamW by understanding why L2 regularization and weight decay diverge in adaptive optimizers.
Examines William Woods's influential 1970 parsing formalism that extended finite-state machines with registers, recursion, and actions.
Explains how batch normalization eliminates internal covariate shift by normalizing layer activations.
Covers L1 regularization (LASSO) in machine learning, covering mathematical foundations, optimization theory, practical implementation.
Explains how special tokens like [CLS], [SEP], [PAD], [MASK], and [UNK] structure transformer inputs, enable classification, handle padding.
Latent Semantic Analysis uses matrix factorization to uncover hidden relationships between terms and documents for retrieval, clustering, and topic discovery.
Covers where subword tokenizers fail: number fragmentation, code identifier splitting, multilingual fertility gaps, emoji edge cases.
Covers POS tagging from tag sets to statistical taggers. Topics include Penn Treebank, Universal Dependencies, emission and transition probabilities.
Multiple linear regression estimates how several predictors relate to an outcome. Covers OLS formulas, assumptions, interpretation, diagnostics, and Python.
Conceptual Dependency maps sentences to primitive actions and roles. Covers Roger Schank's theory, semantic equivalence, inference, and question answering.
Explains how NER identifies people, organizations, and locations using BIO sequence labeling. Topics include entity types, boundary detection, nested entities.
SentencePiece trains BPE and unigram tokenizers directly on raw text. Covers whitespace handling, multilingual vocabularies, and model training.
Andrew Viterbi's 1967 dynamic programming algorithm finds the most likely hidden-state sequence, with applications in HMMs, tagging, and speech recognition.
Train custom tokenizers with HuggingFace, covering corpus preparation, vocabulary sizing, algorithm selection, saving, versioning.
Covers multicollinearity in regression analysis with this practical guide. VIF analysis, correlation matrices, coefficient stability testing.
Covers the BIO tagging scheme for named entity recognition, including BIOES variants, span-to-tag conversion, decoding.
The 1954 Georgetown-IBM demonstration marked an important moment in computational linguistics.
Covers Byte Pair Encoding (BPE), the subword tokenization algorithm powering GPT and modern LLMs. Explains how BPE builds a vocabulary.
Explains how GloVe derives word embeddings from co-occurrence ratios, derives the weighted least squares objective.
Covers Ordinary Least Squares (OLS) regression, including mathematical derivations, matrix formulations, step-by-step examples, and Python implementation.
Covers BM25, the major probabilistic ranking algorithm that changed information retrieval. Explains how BM25 solved TF-IDF's limitations.
Explains how FastText extends Word2Vec with character n-grams to handle out-of-vocabulary words, typos.
Evaluate word embeddings using similarity benchmarks, analogy tests, t-SNE visualization, downstream tasks, and bias detection with WEAT.
Traces Richard Montague's major framework for formal natural language semantics. Explains how Montague Grammar introduced compositionality, intensional logic.
Covers Word2Vec training pipeline: subsampling, learning rate scheduling, minibatch training, convergence monitoring.
Simple linear regression models the relationship between one predictor and an outcome. Covers OLS formulas, assumptions, interpretation, and Python.
Explains how hierarchical softmax reduces word embedding training from O(V) to O(log V) using Huffman-coded binary trees, path probabilities.
Michael Lesk's 1983 algorithm resolves word senses by comparing dictionary definitions with surrounding context. Covers the method, code, and limitations.
Explains how word embeddings solve analogies via vector arithmetic, covering the parallelogram model, 3CosAdd and 3CosMul methods, evaluation datasets.
Explains how negative sampling reduces Word2Vec training from O(V) to O(k), covering the binary classification objective, unigram^0.75 sampling distribution.
Covers R-squared, the coefficient of determination. Explains what R-squared means, how to calculate it, interpret its value.
Explains how Gerard Salton's Vector Space Model and TF-IDF weighting changed information retrieval in 1968.
Examines the CBOW Word2Vec architecture covering context averaging, the objective function, gradient derivation, training dynamics.
Explains how the Skip-gram model trains a neural network to predict context words and learns dense word embeddings.
Examines Noam Chomsky's early 1957 work "Syntactic Structures" that changed linguistics, challenged behaviorism.
Explains how Singular Value Decomposition compresses sparse co-occurrence matrices into dense word embeddings through Latent Semantic Analysis.
Generalized linear models combine link functions with probability distributions. Covers logistic and Poisson regression, maximum likelihood, and Python.
Explains how PMI and PPMI transform raw co-occurrence counts into normalized word association scores.
In 2002, IBM researchers introduced BLEU (Bilingual Evaluation Understudy), revolutionizing machine translation evaluation.
Examines raw, log-scaled, boolean, augmented, and L2-normalized term frequency variants, with sparsity analysis and efficient computation using scikit-learn.
Explains how Firth's principle that words are known by their company underpins distributional semantics.
Create a free account to keep your reading organized, join thoughtful discussions, and get more from every chapter.
Free to join · Takes less than a minute
Why
I created this space so readers can learn together, ask questions, and make sense of difficult ideas.