409 items · Page 9 of 9
Create a free account to keep your reading organized, join thoughtful discussions, and get more from every chapter.
Free to join · Takes less than a minute
Why
I created this space so readers can learn together, ask questions, and make sense of difficult ideas.
Explains how negative sampling reduces Word2Vec training from O(V) to O(k), covering the binary classification objective, unigram^0.75 sampling distribution.
Examines the CBOW Word2Vec architecture covering context averaging, the objective function, gradient derivation, training dynamics.
Explains how the Skip-gram model trains a neural network to predict context words and learns dense word embeddings.
Explains how Singular Value Decomposition compresses sparse co-occurrence matrices into dense word embeddings through Latent Semantic Analysis.
Explains how PMI and PPMI transform raw co-occurrence counts into normalized word association scores.
Examines raw, log-scaled, boolean, augmented, and L2-normalized term frequency variants, with sparsity analysis and efficient computation using scikit-learn.
Explains how Firth's principle that words are known by their company underpins distributional semantics.
Explains how Inverse Document Frequency (IDF) measures word importance across a corpus by weighting rare, discriminative terms higher than common words.
Covers TF-IDF for text representation, including the core formula, variants, normalization, document similarity, feature extraction, sklearn TfidfVectorizer.
Explains how perplexity measures language model quality through cross-entropy and information theory.
Covers BM25, the probabilistic ranking algorithm powering modern search engines. Topics include IDF derivation, term saturation, length normalization, BM25L.
Explains how word-word and word-document co-occurrence matrices encode distributional meaning.
Explains how n-gram language models assign probabilities to word sequences using the chain rule, Markov assumption, and MLE.
Covers smoothing techniques that solve the zero probability problem in n-gram models, from simple add-k smoothing through Good-Turing estimation to Kneser-Ney.
Explains how the Bag of Words model transforms text into numerical vectors through word counting, vocabulary construction, and sparse matrix storage.
Covers sentence boundary detection in NLP, covering the period disambiguation problem, rule-based approaches.
Covers n-gram text representations including bigrams, trigrams, character n-grams, and skip-grams. Topics include extraction techniques.
Split text into words and tokens using whitespace, punctuation handling, and linguistic rules. Topics include NLTK, spaCy, Penn Treebank conventions.
Covers text normalization techniques including Unicode NFC/NFD/NFKC/NFKD forms, case folding vs lowercasing, diacritic removal, and whitespace handling.
Covers regular expressions for text processing, covering metacharacters, quantifiers, lookarounds, and practical NLP patterns.
Covers character encoding fundamentals including ASCII, Unicode, and UTF-8. Detect, fix, and prevent encoding errors like mojibake in your NLP pipelines.
Covers BART's encoder-decoder architecture combining BERT and GPT designs. Examines attention patterns, model configurations, and implementation details.
Covers RAG evaluation with metrics for retrieval quality (Precision@K, NDCG, MRR) and generation faithfulness using the RAGAS framework for AI systems.
Explains how Kaplan scaling laws predict LLM performance from model size, data, and compute.
Examines Mixtral 8x7B's sparse architecture and top-2 expert routing. Explains how MoE models match Llama 2 70B quality.
Create a free account to keep your reading organized, join thoughtful discussions, and get more from every chapter.
Free to join · Takes less than a minute
Why
I created this space so readers can learn together, ask questions, and make sense of difficult ideas.