History of Language AI Articles – Page 2

Michael Brenndoerfer

History of Language AI

Historical perspective on language AI development, from early symbolic systems and parsing algorithms to modern neural approaches.

110 items · Page 2 of 3

Community

Learn with the community

Create a free account to keep your reading organized, join thoughtful discussions, and get more from every chapter.

Keep your place across books and articles
Ask questions and take part in discussions
Read with fewer interruptions

Join the community

Free to join · Takes less than a minute

Why

I created this space so readers can learn together, ask questions, and make sense of difficult ideas.

From readers1 / 12

History of Language AIMachine LearningData, Analytics & AI

ELMo and ULMFiT: Transfer Learning for NLP

Jun 17, 2025·20 min read

Covers ELMo and ULMFiT, the advance methods that established transfer learning for NLP in 2018.

Open notebook

Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI

GPT-1 and GPT-2: Autoregressive Transfer Learning

Jun 15, 2025·18 min read

Covers OpenAI's GPT-1 and GPT-2 models. Explains how autoregressive pretraining with transformers enabled transfer learning across NLP tasks.

Open notebook

Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI

BERT: Bidirectional Pretraining for Language Understanding

Jun 13, 2025·15 min read

Covers BERT (Bidirectional Encoder Representations from Transformers), including masked language modeling, bidirectional context understanding.

Open notebook

Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI

XLNet, RoBERTa, ALBERT

Jun 11, 2025·17 min read

Explains how XLNet, RoBERTa, and ALBERT refined BERT through permutation language modeling, optimized training procedures, and architectural efficiency.

Open notebook

Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI

RLHF Foundations: Learning from Human Preferences in RL

Jun 9, 2025·13 min read

Covers preference-based learning, the framework developed by Christiano et al. in 2017.

Open notebook

History of Language AIMachine LearningData, Analytics & AI

The Transformer: Attention Is All You Need

Jun 7, 2025·20 min read

Covers the Transformer architecture, including self-attention mechanisms, multi-head attention, positional encodings.

Open notebook

History of Language AIMachine LearningData, Analytics & AI

Wikidata: Collaborative Knowledge Base for Language AI

Jun 5, 2025·27 min read

Covers Wikidata, the collaborative multilingual knowledge base launched in 2012. Explains how Wikidata changed structured knowledge representation.

Open notebook

Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI

Subword Tokenization and FastText

Jun 3, 2025·14 min read

Covers FastText and subword tokenization, including character n-gram embeddings, handling out-of-vocabulary words, morphological processing.

Open notebook

Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI

Residual Connections

Jun 1, 2025·15 min read

Covers residual connections, the architectural innovation that solved the vanishing gradient problem in deep networks.

Open notebook

Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI

Google NMT: End-to-End Neural Machine Translation

May 30, 2025·14 min read

Covers Google's transition to neural machine translation in 2016. Explains how GNMT replaced statistical phrase-based methods with end-to-end neural networks.

Open notebook

History of Language AIMachine LearningData, Analytics & AI

Seq2Seq NMT: End-to-End Learning Revolution

May 28, 2025·30 min read

Covers sequence-to-sequence neural machine translation, the 2014 advance that changed translation from statistical pipelines to end-to-end neural models.

Open notebook

Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI

GloVe and Adam Optimizer: Global Word Embeddings

May 26, 2025·25 min read

Covers GloVe (Global Vectors) and the Adam optimizer, two early 2014 developments that changed neural language processing.

Open notebook

Data, Analytics & AIMachine LearningLLM and GenAIHistory of Language AI

Deep Learning for Speech Recognition: The 2012 DNN Results

May 24, 2025·13 min read

The application of deep neural networks to speech recognition in 2012, led by Geoffrey Hinton and his colleagues, marked a major advance.

Open notebook

Machine Learningnatural-language-processingHistory of Language AIneural-networks

Memory Networks: External Memory for Neural QA

May 22, 2025·21 min read

Covers Memory Networks, the 2014 advance that introduced external memory to neural networks.

Open notebook

Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI

Neural Search: Semantic Search with Deep Learning

May 20, 2025·21 min read

Covers neural information retrieval, the advance approach that learned semantic representations for queries and documents.

Open notebook

Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI

Layer Normalization for Sequence Models

May 18, 2025·13 min read

Covers layer normalization, the normalization technique that computes statistics across features for each example.

Open notebook

Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI

Word2Vec: Neural Word Embeddings and Vector Semantics

May 16, 2025·22 min read

Covers word2vec, the advance method for learning dense vector representations of words.

Open notebook

Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI

SQuAD: Reading Comprehension Benchmark

May 14, 2025·16 min read

Covers SQuAD (Stanford Question Answering Dataset), the benchmark that established reading comprehension as a flagship NLP task.

Open notebook

Data, Analytics & AIMachine LearningLLM and GenAIHistory of Language AI

WaveNet: Neural Audio Generation Revolution

May 12, 2025·16 min read

DeepMind's WaveNet changed text-to-speech synthesis in 2016 by generating raw audio waveforms directly using neural networks.

Open notebook

History of Language AIData, Analytics & AIMachine Learning

IBM Watson on Jeopardy!

May 10, 2025·11 min read

Examines IBM Watson's historic victory on Jeopardy! in February 2011, examining the system's architecture, multi-hypothesis answer generation.

Open notebook

Data, Analytics & AIMachine LearningHistory of Language AI

Freebase: Collaborative Knowledge Graph

May 8, 2025·19 min read

Freebase introduced a collaborative knowledge graph for representing entities and relationships. Covers its schema, community editing, applications, and legacy.

Open notebook

Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI

Latent Dirichlet Allocation

May 6, 2025·20 min read

Covers Latent Dirichlet Allocation (LDA), the advance Bayesian probabilistic model that changed topic modeling.

Open notebook

History of Language AIData, Analytics & AIMachine Learning

Neural Probabilistic LLM: Distributed Word Representations

May 4, 2025·12 min read

Examines Yoshua Bengio's early 2003 Neural Probabilistic Language Model that changed NLP by learning dense, continuous word embeddings.

Open notebook

Data, Analytics & AIMachine LearningLLM and GenAIHistory of Language AI

PropBank: Semantic Role Labeling and Proposition Bank

May 2, 2025·20 min read

In 2005, the PropBank project at the University of Pennsylvania added semantic role labels to the Penn Treebank.

Open notebook

History of Language AIData, Analytics & AIMachine Learning

Statistical Parsers: From Rules to Probabilistic Models

Apr 30, 2025·18 min read

Traces statistical parsing's major shift from rule-based to data-driven approaches. Explains how Michael Collins's 1997 parser.

Open notebook

History of Language AI

Phrase-Based SMT and Minimum Error Rate Training

Apr 28, 2025·20 min read

How phrase-based translation (2003) extended IBM statistical MT to phrase-level learning, capturing idioms and collocations.

Open notebook

History of Language AI

Maximum Entropy & Support Vector Machines in NLP

Apr 26, 2025·23 min read

How Maximum Entropy models and Support Vector Machines changed NLP in 1996 by enabling flexible feature integration for sequence labeling, text classification.

Open notebook

Data, Analytics & AIMachine LearningLLM and GenAIHistory of Language AI

FrameNet: A Computational Resource for Frame Semantics

Apr 24, 2025·25 min read

In 1998, Charles Fillmore's FrameNet project at ICSI Berkeley released the first large-scale computational resource based on frame semantics.

Open notebook

History of Language AIData, Analytics & AIMachine Learning

Chinese Room Argument: Syntax, Semantics and Computation

Apr 22, 2025·22 min read

Examines John Searle's influential 1980 thought experiment challenging strong AI.

Open notebook

History of Language AIData, Analytics & AIMachine Learning

Augmented Transition Networks

Apr 20, 2025·13 min read

Examines William Woods's influential 1970 parsing formalism that extended finite-state machines with registers, recursion, and actions.

Open notebook

Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI

Latent Semantic Analysis and Topic Models

Apr 18, 2025·21 min read

Latent Semantic Analysis uses matrix factorization to uncover hidden relationships between terms and documents for retrieval, clustering, and topic discovery.

Open notebook

History of Language AIData, Analytics & AIMachine Learning

Conceptual Dependency

Apr 16, 2025·19 min read

Conceptual Dependency maps sentences to primitive actions and roles. Covers Roger Schank's theory, semantic equivalence, inference, and question answering.

Open notebook

History of Language AIData, Analytics & AIMachine Learning

Viterbi Algorithm: Dynamic Programming Foundation

Apr 14, 2025·21 min read

Andrew Viterbi's 1967 dynamic programming algorithm finds the most likely hidden-state sequence, with applications in HMMs, tagging, and speech recognition.

Open notebook

Data, Analytics & AIMachine LearningHistory of Language AI

Georgetown-IBM Machine Translation Demonstration

Apr 12, 2025·16 min read

The 1954 Georgetown-IBM demonstration marked an important moment in computational linguistics.

Open notebook

Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI

BM25: Probabilistic Ranking for Search

Apr 10, 2025·12 min read

Covers BM25, the major probabilistic ranking algorithm that changed information retrieval. Explains how BM25 solved TF-IDF's limitations.

Open notebook

History of Language AIMachine LearningData, Analytics & AI

Montague Semantics and Compositional Meaning

Apr 8, 2025·27 min read

Traces Richard Montague's major framework for formal natural language semantics. Explains how Montague Grammar introduced compositionality, intensional logic.

Open notebook

History of Language AIData, Analytics & AIMachine Learning

Lesk Algorithm for Word Sense Disambiguation

Apr 6, 2025·22 min read

Michael Lesk's 1983 algorithm resolves word senses by comparing dictionary definitions with surrounding context. Covers the method, code, and limitations.

Open notebook

History of Language AIMachine LearningData, Analytics & AI

Vector Space Model & TF-IDF: Foundation of Modern Search

Apr 4, 2025·14 min read

Explains how Gerard Salton's Vector Space Model and TF-IDF weighting changed information retrieval in 1968.

Open notebook

History of Language AIData, Analytics & AIMachine Learning

Chomsky's Syntactic Structures

Apr 2, 2025·21 min read

Examines Noam Chomsky's early 1957 work "Syntactic Structures" that changed linguistics, challenged behaviorism.

Open notebook

Data, Analytics & AIMachine LearningLLM and GenAIHistory of Language AI

BLEU Metric: Automatic Evaluation for Machine Translation

Mar 31, 2025·23 min read

In 2002, IBM researchers introduced BLEU (Bilingual Evaluation Understudy), revolutionizing machine translation evaluation.

Open notebook

Data, Analytics & AIMachine LearningLLM and GenAIHistory of Language AI

Conditional Random Fields: History CRF

Mar 29, 2025·24 min read

Conditional Random Fields model structured predictions without assuming independent labels. Covers the 2001 paper, feature design, inference, and later uses.

Open notebook

Data, Analytics & AIMachine LearningLLM and GenAIHistory of Language AI

From Symbolic Rules to Statistical Learning

Mar 27, 2025·21 min read

Natural language processing underwent a fundamental shift from symbolic rules to statistical learning.

Open notebook

Data, Analytics & AIMachine LearningLLM and GenAIHistory of Language AI

Shannon's N-gram Model

Mar 25, 2025·13 min read

Claude Shannon's 1948 work on information theory introduced n-gram models, one of the most core concepts in natural language processing.

Open notebook

Data, Analytics & AIMachine LearningLLM and GenAIHistory of Language AI

The Turing Test: Imitation and Machine Intelligence

Mar 23, 2025·14 min read

In 1950, Alan Turing proposed a deceptively simple test for machine intelligence, originally called the Imitation Game.

Open notebook

Data, Analytics & AIMachine LearningLLM and GenAIHistory of Language AI

ELIZA: The First Conversational AI Program

Mar 21, 2025·14 min read

Joseph Weizenbaum's ELIZA, created in 1966, became the first computer program to hold something resembling a conversation.

Open notebook

Data, Analytics & AIMachine LearningLLM and GenAIHistory of Language AI

Hidden Markov Models: Statistical Speech Recognition

Mar 19, 2025·22 min read

Hidden Markov Models changed speech recognition in the 1970s by introducing a clever probabilistic approach.

Open notebook

Data, Analytics & AIMachine LearningLLM and GenAIHistory of Language AI

The Perceptron: Foundation of Modern Neural Networks

Mar 17, 2025·25 min read

In 1958, Frank Rosenblatt created the perceptron at Cornell Aeronautical Laboratory, the first artificial neural network.

Open notebook

Data, Analytics & AIMachine LearningLLM and GenAIHistory of Language AI

SHRDLU: Understanding Language Through Action

Mar 15, 2025·10 min read

In 1968, Terry Winograd's SHRDLU system demonstrated a major approach to natural language understanding by grounding language in a simulated blocks world.

Open notebook