https://mbrenndoerfer.com/writing/zero-optimization-stages-optimizer-gradient-parameter-partitioning/zero-optimization-optimizer-gradient-and-parameter-partitioning2026-01-21T21:43:02.000Zhttps://mbrenndoerfer.com/writing/zero-optimization-stages-optimizer-gradient-parameter-partitioning/the-memory-anatomy-of-training2026-01-21T21:43:02.000Zhttps://mbrenndoerfer.com/writing/zero-optimization-stages-optimizer-gradient-parameter-partitioning/zero-stage-1-optimizer-state-partitioning2026-01-21T21:43:02.000Zhttps://mbrenndoerfer.com/writing/zero-optimization-stages-optimizer-gradient-parameter-partitioning/zero-stage-2-gradient-partitioning2026-01-21T21:43:02.000Zhttps://mbrenndoerfer.com/writing/zero-optimization-stages-optimizer-gradient-parameter-partitioning/zero-stage-3-parameter-partitioning2026-01-21T21:43:02.000Zhttps://mbrenndoerfer.com/writing/zero-optimization-stages-optimizer-gradient-parameter-partitioning/memory-savings-across-stages2026-01-21T21:43:02.000Zhttps://mbrenndoerfer.com/writing/zero-optimization-stages-optimizer-gradient-parameter-partitioning/zero-offload-and-zero-infinity2026-01-21T21:43:02.000Zhttps://mbrenndoerfer.com/writing/zero-optimization-stages-optimizer-gradient-parameter-partitioning/zero-and-communication-efficiency2026-01-21T21:43:02.000Zhttps://mbrenndoerfer.com/writing/zero-optimization-stages-optimizer-gradient-parameter-partitioning/worked-example-training-state-partitioning2026-01-21T21:43:02.000Zhttps://mbrenndoerfer.com/writing/zero-optimization-stages-optimizer-gradient-parameter-partitioning/practical-considerations2026-01-21T21:43:02.000Zhttps://mbrenndoerfer.com/writing/zero-optimization-stages-optimizer-gradient-parameter-partitioning/zero-in-the-pytorch-ecosystem-fsdp2026-01-21T21:43:02.000Zhttps://mbrenndoerfer.com/writing/zero-optimization-stages-optimizer-gradient-parameter-partitioning/limitations-and-impact2026-01-21T21:43:02.000Zhttps://mbrenndoerfer.com/writing/zero-optimization-stages-optimizer-gradient-parameter-partitioning/summary2026-01-21T21:43:02.000Zhttps://mbrenndoerfer.com/writing/zero-optimization-stages-optimizer-gradient-parameter-partitioning/quiz2026-01-21T21:43:02.000Z