712 items · Page 15 of 15
Create a free account to keep your reading organized, join thoughtful discussions, and get more from every chapter.
Free to join · Takes less than a minute
Why
I created this space so readers can learn together, ask questions, and make sense of difficult ideas.
Conditional Random Fields model structured predictions without assuming independent labels. Covers the 2001 paper, feature design, inference, and later uses.
Explains how Inverse Document Frequency (IDF) measures word importance across a corpus by weighting rare, discriminative terms higher than common words.
Covers TF-IDF for text representation, including the core formula, variants, normalization, document similarity, feature extraction, sklearn TfidfVectorizer.
Natural language processing underwent a fundamental shift from symbolic rules to statistical learning.
Explains how perplexity measures language model quality through cross-entropy and information theory.
Covers BM25, the probabilistic ranking algorithm powering modern search engines. Topics include IDF derivation, term saturation, length normalization, BM25L.
Claude Shannon's 1948 work on information theory introduced n-gram models, one of the most core concepts in natural language processing.
Explains how word-word and word-document co-occurrence matrices encode distributional meaning.
Explains how n-gram language models assign probabilities to word sequences using the chain rule, Markov assumption, and MLE.
In 1950, Alan Turing proposed a deceptively simple test for machine intelligence, originally called the Imitation Game.
Covers smoothing techniques that solve the zero probability problem in n-gram models, from simple add-k smoothing through Good-Turing estimation to Kneser-Ney.
Explains how the Bag of Words model transforms text into numerical vectors through word counting, vocabulary construction, and sparse matrix storage.
Joseph Weizenbaum's ELIZA, created in 1966, became the first computer program to hold something resembling a conversation.
Covers sentence boundary detection in NLP, covering the period disambiguation problem, rule-based approaches.
Covers n-gram text representations including bigrams, trigrams, character n-grams, and skip-grams. Topics include extraction techniques.
Hidden Markov Models changed speech recognition in the 1970s by introducing a clever probabilistic approach.
Split text into words and tokens using whitespace, punctuation handling, and linguistic rules. Topics include NLTK, spaCy, Penn Treebank conventions.
Covers text normalization techniques including Unicode NFC/NFD/NFKC/NFKD forms, case folding vs lowercasing, diacritic removal, and whitespace handling.
In 1958, Frank Rosenblatt created the perceptron at Cornell Aeronautical Laboratory, the first artificial neural network.
Covers regular expressions for text processing, covering metacharacters, quantifiers, lookarounds, and practical NLP patterns.
Covers character encoding fundamentals including ASCII, Unicode, and UTF-8. Detect, fix, and prevent encoding errors like mojibake in your NLP pipelines.
In 1968, Terry Winograd's SHRDLU system demonstrated a major approach to natural language understanding by grounding language in a simulated blocks world.
Bernard Widrow and Marcian Hoff built MADALINE at Stanford in 1962, taking neural networks beyond the perceptron's limitations.
In 1991, IBM researchers changed machine translation by introducing the first broad statistical approach.
In 1995, RNNs changed sequence processing by introducing neural networks with memory: connections.
Hochreiter and Schmidhuber introduced LSTMs in 1997 to address vanishing gradients. Covers gated memory, long context, and later sequence models.
In the 1980s, neural networks hit a wall: nobody knew how to train deep models. That changed when Rumelhart, Hinton.
In the mid-1990s, Princeton University released WordNet, a major lexical database.
In 1988, Yann LeCun introduced Convolutional Neural Networks at Bell Labs, forever changing how machines process visual information.
In 1987, Slava Katz solved one of statistical language modeling's biggest problems. When your model encounters word sequences it has never seen before.
In 1987, Alex Waibel introduced Time Delay Neural Networks, a major architecture that changed how neural networks process sequential data.
Covers OpenAI's ChatGPT release in 2022, including the conversational interface, RLHF training approach, safety measures.
Covers XLM (Cross-lingual Language Model) introduced by Facebook AI Research in 2019. Explains how cross-lingual pretraining.
Covers long context language models introduced in 2024. Explains how models achieved 1M+ token context windows through efficient attention mechanisms.
In 2004, ROUGE and METEOR addressed critical limitations in BLEU's evaluation approach.
The Penn Treebank established large tagged corpora for statistical NLP, enabling comparable research in part-of-speech tagging and syntactic parsing.
Covers BART's encoder-decoder architecture combining BERT and GPT designs. Examines attention patterns, model configurations, and implementation details.
Covers RAG evaluation with metrics for retrieval quality (Precision@K, NDCG, MRR) and generation faithfulness using the RAGAS framework for AI systems.
Explains how Kaplan scaling laws predict LLM performance from model size, data, and compute.
Examines Mixtral 8x7B's sparse architecture and top-2 expert routing. Explains how MoE models match Llama 2 70B quality.
Create a free account to keep your reading organized, join thoughtful discussions, and get more from every chapter.
Free to join · Takes less than a minute
Why
I created this space so readers can learn together, ask questions, and make sense of difficult ideas.