https://mbrenndoerfer.com/writing/cross-entropy-loss-language-models-information-theory/cross-entropy-loss2026-02-21T14:30:00.000Zhttps://mbrenndoerfer.com/writing/cross-entropy-loss-language-models-information-theory/information-theory-foundations2026-02-21T14:30:00.000Zhttps://mbrenndoerfer.com/writing/cross-entropy-loss-language-models-information-theory/defining-cross-entropy2026-02-21T14:30:00.000Zhttps://mbrenndoerfer.com/writing/cross-entropy-loss-language-models-information-theory/cross-entropy-and-perplexity2026-02-21T14:30:00.000Zhttps://mbrenndoerfer.com/writing/cross-entropy-loss-language-models-information-theory/bits-per-character2026-02-21T14:30:00.000Zhttps://mbrenndoerfer.com/writing/cross-entropy-loss-language-models-information-theory/binary-vs-categorical-cross-entropy2026-02-21T14:30:00.000Zhttps://mbrenndoerfer.com/writing/cross-entropy-loss-language-models-information-theory/gradient-behavior-and-why-cross-entropy-works2026-02-21T14:30:00.000Zhttps://mbrenndoerfer.com/writing/cross-entropy-loss-language-models-information-theory/training-dynamics-and-loss-curves2026-02-21T14:30:00.000Zhttps://mbrenndoerfer.com/writing/cross-entropy-loss-language-models-information-theory/worked-example2026-02-21T14:30:00.000Zhttps://mbrenndoerfer.com/writing/cross-entropy-loss-language-models-information-theory/code-implementation2026-02-21T14:30:00.000Zhttps://mbrenndoerfer.com/writing/cross-entropy-loss-language-models-information-theory/numerical-stability-in-practice2026-02-21T14:30:00.000Zhttps://mbrenndoerfer.com/writing/cross-entropy-loss-language-models-information-theory/implementation-parameters2026-02-21T14:30:00.000Zhttps://mbrenndoerfer.com/writing/cross-entropy-loss-language-models-information-theory/limitations-and-impact2026-02-21T14:30:00.000Zhttps://mbrenndoerfer.com/writing/cross-entropy-loss-language-models-information-theory/summary2026-02-21T14:30:00.000Zhttps://mbrenndoerfer.com/writing/cross-entropy-loss-language-models-information-theory/quiz2026-02-21T14:30:00.000Z