Software Engineering

Insights on software development, engineering practices, architecture patterns, and the craft of building robust, scalable systems.

405 items · Page 7 of 9

Community

Learn with the community

Create a free account to keep your reading organized, join thoughtful discussions, and get more from every chapter.

  • Keep your place across books and articles
  • Ask questions and take part in discussions
  • Read with fewer interruptions

Free to join · Takes less than a minute

Why

I created this space so readers can learn together, ask questions, and make sense of difficult ideas.

Michael Brenndoerfer
From readers1 / 12
AI Agent HandbookMachine LearningSoftware Engineering

Communication Between Agents: Message Formats, Protocols

Jul 27, 2025·20 min read

Explains how AI agents exchange information and coordinate actions through structured messages, communication patterns like pub-sub and request-response.

Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI

InstructGPT and RLHF: Aligning LLMs with Human Preferences

Jul 25, 2025·16 min read

Covers OpenAI's InstructGPT research from 2022, including the three-stage RLHF training process, supervised fine-tuning, reward modeling.

Open notebook
AI Agent HandbookMachine LearningData, Analytics & AISoftware Engineering

Agents Working Together: Multi-Agent Systems

Jul 25, 2025·17 min read

Explains how multiple AI agents collaborate through specialization, parallel processing, and coordination.

Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI

The Pile: Open-Source Training Dataset for LLMs

Jul 23, 2025·14 min read

Covers EleutherAI's The Pile, the early 825GB open-source dataset that broadened access to high-quality training data for large language models.

Open notebook
AI Agent HandbookMachine LearningSoftware Engineering

AI Agent Planning: Meeting Scheduling and Summaries

Jul 23, 2025·10 min read

See how AI agents use planning to handle complex, multi-step tasks. Topics include task decomposition, sequential execution.

Open notebook
Data, Analytics & AISoftware EngineeringLLM and GenAI

Adapting LLMs: Context Injection vs. Fine-Tuning

Jul 22, 2025·15 min read

Covers choosing the right approach for your LLM project: using pre-trained models as-is, enhancing them with context injection and RAG.

Read article
Data, Analytics & AISoftware EngineeringLLM and GenAI

Building AI Agents with LangChain and LangGraph: Part 1

Jul 21, 2025·6 min read

Covers the core concepts of LLM workflows: connecting language models to tools, handling responses, and building intelligent systems.

Open notebook
AI Agent HandbookMachine LearningSoftware Engineering

Plan and Execute: Turning Agent Plans into Action

Jul 21, 2025·12 min read

Explains how AI agents execute multi-step plans sequentially, handle failures gracefully, and adapt when things go wrong.

Open notebook
Data, Analytics & AISoftware EngineeringMachine Learning

Simulating stock market returns using Monte Carlo

Jul 19, 2025·13 min read

Use Monte Carlo simulation to model and analyze stock market returns, estimate future performance.

Open notebook
AI Agent HandbookMachine LearningData, Analytics & AISoftware Engineering

Task Decomposition for AI Agents

Jul 19, 2025·13 min read

Explains how AI agents break down complex goals into manageable subtasks. Topics include task decomposition strategies, sequential vs parallel tasks.

Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningMachine Learning from Scratch

XGBoost: Extreme Gradient Boosting

Jul 18, 2025·78 min read

Covers XGBoost (eXtreme Gradient Boosting), including second-order Taylor expansion, regularization techniques, split gain optimization, ranking loss functions.

Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI

Scaling Laws for Neural LLMs

Jul 17, 2025·20 min read

Covers the 2020 scaling laws discovered by Kaplan et al. Explains how power-law relationships predict model performance from scale.

Open notebook
AI Agent HandbookMachine LearningSoftware Engineering

Environment Boundaries and Constraints

Jul 17, 2025·18 min read

Define what your AI agent can and cannot do through access controls, action policies, rate limits, and scope boundaries.

Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningMachine Learning from Scratch

SHAP (SHapley Additive exPlanations): Model Interpretability

Jul 15, 2025·55 min read

Covers SHAP values covering mathematical foundations, feature attribution, and practical implementations for explaining any machine learning model.

Open notebook
AI Agent HandbookMachine LearningSoftware Engineering

Perception and Action: How AI Agents Sense

Jul 15, 2025·13 min read

Explains how AI agents perceive their environment through inputs, tool outputs, and memory.

Open notebook
AI Agent HandbookMachine LearningSoftware Engineering

Defining the Agent's Environment: Where AI Agents Operate

Jul 13, 2025·10 min read

Explains what an environment means for AI agents, from digital assistants to physical robots. Explains how environment shapes perception, actions.

Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningMachine Learning from Scratch

LightGBM: Fast Gradient Boosting

Jul 12, 2025·52 min read

Covers LightGBM gradient boosting framework, including leaf-wise tree growth, histogram-based binning, GOSS sampling, exclusive feature bundling.

Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI

FlashAttention: IO-Aware Attention for Long Contexts

Jul 11, 2025·12 min read

Covers FlashAttention introduced in 2022. Explains how IO-aware attention computation enabled 2-4x speedup and 5-10x memory reduction.

Open notebook
AI Agent HandbookMachine LearningSoftware Engineering

Managing State Across Interactions: Agent State Lifecycle

Jul 11, 2025·13 min read

Explains how AI agents maintain continuity across sessions with ephemeral, session, and persistent state management.

Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningMachine Learning from Scratch

CatBoost: Categorical Boosting

Jul 9, 2025·42 min read

Covers CatBoost (Categorical Boosting), including categorical feature handling, target statistics, symmetric trees, ordered boosting, regularization techniques.

Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI

CLIP: Contrastive Pretraining for Vision and Language

Jul 9, 2025·19 min read

Covers OpenAI's CLIP, the early vision-language model that enables zero-shot image classification through contrastive learning.

Open notebook
AI Agent HandbookSoftware EngineeringMachine Learning

Designing the Agent's Brain

Jul 9, 2025·14 min read

Structure AI agents with clear architecture patterns. Build organized agent loops, decision logic.

Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI

Instruction Tuning for Language Models

Jul 7, 2025·14 min read

Covers instruction tuning introduced in 2021. Explains how fine-tuning on diverse instruction-response pairs changed language models.

Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningMachine Learning from Scratch

Isolation Forest: Unsupervised Anomaly Detection

Jul 6, 2025·48 min read

Covers Isolation Forest covering unsupervised anomaly detection, path length calculations, harmonic numbers, anomaly scoring.

Open notebook
AI Agent HandbookMachine LearningSoftware Engineering

Implementing Memory in Our Agent

Jul 5, 2025·17 min read

Build a complete AI agent memory system combining conversation history and persistent knowledge storage.

Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningLanguage AI Handbook

Position Interpolation

Jul 3, 2025·49 min read

Explains how Position Interpolation extends transformer context windows by scaling position indices to stay within training distributions.

Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningMachine Learning from Scratch

Boosted Trees: Gradient Boosting Algorithm & Implementation

Jul 3, 2025·44 min read

Covers boosted trees and gradient boosting, covering ensemble learning, loss functions, sequential error correction, and scikit-learn implementation.

Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI

DALL·E 2: Diffusion and CLIP-Guided Image Generation

Jul 3, 2025·14 min read

Covers OpenAI's DALL·E 2, the major text-to-image generation model that combined CLIP-guided diffusion with high-quality image synthesis.

Open notebook
AI Agent HandbookMachine LearningSoftware Engineering

Long-Term Knowledge Storage and Retrieval

Jul 3, 2025·14 min read

Explains how AI agents store and retrieve information across sessions using vector databases, embeddings, and semantic search.

Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningLanguage AI Handbook

Context Length Challenges

Jul 2, 2025·39 min read

Examines the fundamental barriers limiting transformer context windows, including quadratic memory scaling, position encoding extrapolation failures.

Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI

Codex: AI-Assisted Code Generation

Jul 1, 2025·18 min read

Codex adapted GPT-3 for code completion and generation. Covers its training data, natural-language prompts, benchmark results, and role in developer tools.

Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningMachine Learning from Scratch

Random Forest: Ensemble Learning

Jun 30, 2025·43 min read

Covers Random Forest covering ensemble learning, bootstrap sampling, random feature selection, bias-variance tradeoff, and implementation in scikit-learn.

Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI

DALL·E: Text-to-Image Generation

Jun 29, 2025·12 min read

Covers OpenAI's DALL·E, the early text-to-image generation model that extended transformer architectures to multimodal tasks.

Open notebook
AI Agent HandbookMachine LearningSoftware Engineering

Adding a Calculator Tool to Your AI Agent

Jun 29, 2025·15 min read

Build a working calculator tool for your AI agent from scratch. Topics include the complete workflow from Python function to tool integration.

Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningMachine Learning from Scratch

CART Decision Trees: Classification and Regression Trees

Jun 27, 2025·45 min read

Covers CART (Classification and Regression Trees), including mathematical foundations, Gini impurity, variance reduction.

Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI

GPT-3 and In-Context Learning

Jun 27, 2025·21 min read

GPT-3 showed that scaling autoregressive transformers could enable in-context and few-shot learning. Covers its architecture, data, prompting, and limitations.

Open notebook
AI Agent HandbookMachine LearningSoftware Engineering

Using a LLM in Code: API Integration & Implementation

Jun 27, 2025·15 min read

Call language models from Python code, including GPT-5, Claude Sonnet 4.5, and Gemini 2.5. Covers API integration, error handling.

Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI

T5 and Text-to-Text Framework

Jun 25, 2025·19 min read

Covers Google's T5 (Text-to-Text Transfer Transformer) introduced in 2019. Explains how the text-to-text framework unified diverse NLP tasks.

Open notebook
AI Agent HandbookMachine LearningSoftware Engineering

Designing Simple Tool Interfaces

Jun 25, 2025·14 min read

Design effective tool interfaces for AI agents, from basic function definitions to multi-tool orchestration.

Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningMachine Learning from Scratch

Logistic Regression: Math & Python

Jun 24, 2025·46 min read

Covers logistic regression covering mathematical foundations, the logistic function, optimization algorithms, and practical implementation.

Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningLanguage AI Handbook

Longformer: Efficient Attention for Long Documents

Jun 23, 2025·55 min read

Explains how Longformer combines sliding window and global attention to process documents of 4,096+ tokens with O(n) complexity instead of O(n²).

Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningLanguage AI Handbook

Sparse Attention Patterns: Local, Strided

Jun 22, 2025·61 min read

Implement sparse attention patterns including local windows, strided attention, and block-sparse methods.

Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningMachine Learning from Scratch

Poisson Regression: Count Data Modeling with Math & Python

Jun 21, 2025·49 min read

Covers Poisson regression for count data analysis. Topics include mathematical foundations, maximum likelihood estimation, rate ratio interpretation.

Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI

Transformer-XL: Extending Transformers to Long Sequences

Jun 21, 2025·6 min read

Transformer-XL extends context with segment recurrence and relative positions. Covers its architecture, efficiency, training method, and influence.

Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningMachine Learning from Scratch

Spline Regression: Non-Linear Modeling with Math & Python

Jun 18, 2025·65 min read

Spline regression models nonlinear relationships with piecewise polynomials. Covers B-splines, knot selection, natural cubic splines, and Python.

Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningMachine Learning from Scratch

Multinomial Logistic Regression: Math & Python

Jun 15, 2025·48 min read

Covers multinomial logistic regression covering mathematical foundations, softmax function, coefficient estimation.

Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI

GPT-1 and GPT-2: Autoregressive Transfer Learning

Jun 15, 2025·18 min read

Covers OpenAI's GPT-1 and GPT-2 models. Explains how autoregressive pretraining with transformers enabled transfer learning across NLP tasks.

Open notebook
Data, Analytics & AISoftware EngineeringMachine LearningHistory of Language AI

BERT: Bidirectional Pretraining for Language Understanding

Jun 13, 2025·15 min read

Covers BERT (Bidirectional Encoder Representations from Transformers), including masked language modeling, bidirectional context understanding.

Open notebook
Community

Learn with the community

Create a free account to keep your reading organized, join thoughtful discussions, and get more from every chapter.

  • Keep your place across books and articles
  • Ask questions and take part in discussions
  • Read with fewer interruptions

Free to join · Takes less than a minute

Why

I created this space so readers can learn together, ask questions, and make sense of difficult ideas.

Michael Brenndoerfer
From readers1 / 12