409 items · Page 8 of 9
Create a free account to keep your reading organized, join thoughtful discussions, and get more from every chapter.
Free to join · Takes less than a minute
Why
I created this space so readers can learn together, ask questions, and make sense of difficult ideas.
Explains how bidirectional LSTMs and GRUs provide full-sequence context by processing text forward and backward, enabling strong sequence labeling.
Explains how Bahdanau attention solves the encoder-decoder bottleneck with dynamic context vectors, softmax alignment.
Examines Luong attention variants including dot product, general, and concat scoring. Topics include global vs local attention, input-feeding.
Explains how copy mechanisms enable seq2seq models to handle out-of-vocabulary words.
Explains how attention mechanisms solve the encoder-decoder bottleneck through soft lookup, alignment scores, and dynamic context vectors built step by step.
Covers the encoder-decoder framework for sequence-to-sequence learning. Topics include context vectors, teacher forcing, BLEU evaluation.
Gated Recurrent Units use reset and update gates to manage sequence memory. Covers GRU equations, implementation, gradient flow, and comparison with LSTMs.
Stacked RNNs build hierarchical representations across recurrent layers. Covers depth-width tradeoffs, variational dropout, ELMo, and practical depth limits.
How LSTMs solve the vanishing gradient problem through the cell state gradient highway, forget gate modulation, and the residual connection analogy.
Covers LSTM architecture including cell state, gate mechanisms, and information flow. Explains how LSTMs solve vanishing gradients.
Explains how BPTT unrolls RNNs for gradient computation, derives the chain rule through time, and implements truncated BPTT with detach() in PyTorch.
Covers mathematics behind LSTM gates including forget, input, output gates, and cell state updates.
Explains why gradient magnitudes collapse exponentially in RNNs, making long-range dependencies impossible to learn without architectural changes like LSTMs.
Covers RNN architecture from Elman equations to hidden state memory. Topics include parameter sharing, unrolling, sequence modeling patterns.
Covers backpropagation from computational graphs to PyTorch autograd. Topics include chain rule derivation, gradient flow through activations.
Covers chunking (shallow parsing) to identify noun phrases, verb phrases, and prepositional phrases using IOB tagging, regex patterns.
Explains how Hidden Markov Models use transition and emission probabilities to solve sequence labeling tasks like POS tagging, with Python implementation.
Covers CRFs for sequence labeling, from log-linear models to feature functions and the forward algorithm.
Explains how finite-state transducers and neural sequence-to-sequence models perform morphological analysis and generation.
Explains how words decompose into morphemes, why inflection and derivation differ, and how morphology explains NLP sparsity, OOV rates, and tokenizer failures.
Covers neural network loss functions covering MSE, cross-entropy, label smoothing, focal loss, KL divergence, and contrastive losses for NLP tasks.
Covers Conditional Random Field training with the forward-backward algorithm, gradient computation, and L-BFGS optimization for sequence labeling tasks.
Covers SGD for training neural networks: mini-batch updates, learning rate selection, cosine annealing, warmup, Nesterov momentum, and loss landscape geometry.
Explains how MLPs stack layers to solve problems no linear model can. Topics include hidden layers, weight matrices, forward pass, universal approximation.
Covers linear classifiers, from weighted dot products and decision boundary geometry to softmax for multiclass problems, gradient descent training.
Covers dropout regularization: inverted dropout scaling, MC dropout uncertainty, spatial dropout for sequences, and dropout in transformers.
Covers Viterbi algorithm for finding optimal tag sequences in HMMs. Topics include dynamic programming, backpointer tracking, log-space computation.
Explains why weight initialization matters for training neural networks. Topics include Xavier and He initialization, orthogonal init, BERT and GPT schemes.
Covers Adam optimizer: first and second moment estimates, bias correction, adaptive learning rates, hyperparameter tuning.
Momentum smooths gradient updates with a velocity term. Covers Polyak momentum, Nesterov acceleration, damped oscillations, and optimization tradeoffs.
Explains how gradient clipping prevents training instability by capping gradient magnitudes. Topics include clip by value vs clip by norm strategies.
Covers every major activation function from sigmoid to GELU. Topics include saturation, dying ReLU, gradient flow analysis.
Covers AdamW by understanding why L2 regularization and weight decay diverge in adaptive optimizers.
Explains how batch normalization eliminates internal covariate shift by normalizing layer activations.
Explains how special tokens like [CLS], [SEP], [PAD], [MASK], and [UNK] structure transformer inputs, enable classification, handle padding.
Covers where subword tokenizers fail: number fragmentation, code identifier splitting, multilingual fertility gaps, emoji edge cases.
Covers POS tagging from tag sets to statistical taggers. Topics include Penn Treebank, Universal Dependencies, emission and transition probabilities.
Explains how NER identifies people, organizations, and locations using BIO sequence labeling. Topics include entity types, boundary detection, nested entities.
SentencePiece trains BPE and unigram tokenizers directly on raw text. Covers whitespace handling, multilingual vocabularies, and model training.
Train custom tokenizers with HuggingFace, covering corpus preparation, vocabulary sizing, algorithm selection, saving, versioning.
Covers the BIO tagging scheme for named entity recognition, including BIOES variants, span-to-tag conversion, decoding.
Covers Byte Pair Encoding (BPE), the subword tokenization algorithm powering GPT and modern LLMs. Explains how BPE builds a vocabulary.
Explains how GloVe derives word embeddings from co-occurrence ratios, derives the weighted least squares objective.
Explains how FastText extends Word2Vec with character n-grams to handle out-of-vocabulary words, typos.
Evaluate word embeddings using similarity benchmarks, analogy tests, t-SNE visualization, downstream tasks, and bias detection with WEAT.
Covers Word2Vec training pipeline: subsampling, learning rate scheduling, minibatch training, convergence monitoring.
Explains how hierarchical softmax reduces word embedding training from O(V) to O(log V) using Huffman-coded binary trees, path probabilities.
Explains how word embeddings solve analogies via vector arithmetic, covering the parallelogram model, 3CosAdd and 3CosMul methods, evaluation datasets.
Create a free account to keep your reading organized, join thoughtful discussions, and get more from every chapter.
Free to join · Takes less than a minute
Why
I created this space so readers can learn together, ask questions, and make sense of difficult ideas.