Architecture Methods › Visualizations

Michael BrenndoerferFebruary 22, 20264 min read
Language AI HandbookAll chapters
Part 46 of 67Continual Learning
Chapter 355Architecture Methods › Visualizations
Reading section 10 of 13 · Visualizations · 4 minSection 10 of 13 · 4 min

Book contents

Language AI Handbook67 parts · 513 chaptersView the whole book →
Active chapter 355 of 513

Section 10 of 13 · 4 min read

Architecture Methods › VisualizationsOne section · Sections in this chapter10 / 13 ↓
Sections in this chapter13
Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

Out[11]:
Visualization
Line chart across tasks 1 through 10. Progressive-network capacity remains at 1.0, PackNet capacity halves each task to about 0.002 by task 10, and HAT capacity declines more gradually to about 0.04. Circle, square, and triangle markers distinguish the three methods.
Illustrative capacity available to each new task under three allocation rules across ten sequential tasks: one full column per progressive-network task, 50% geometric pruning for PackNet, and a 30% claim on HAT''s remaining units. Progressive networks preserve full per-task capacity by growing memory, while the two fixed-budget methods leave successively less capacity for later tasks.
Out[12]:
Visualization
Grouped bars at 1, 5, and 10 tasks. Progressive-network cost rises from 1x to 5x and 10x, PackNet remains at 1x, and HAT rises slightly from 1.01x to 1.05x and 1.10x.
Illustrative total parameter cost normalized to a single-task baseline at three task milestones. The model assumes one full column per progressive-network task, a fixed PackNet backbone, and HAT mask storage equal to 1% of the baseline per task; under these assumptions, progressive networks reach 10x cost at ten tasks while PackNet and HAT remain near 1x.
Out[13]:
Visualization
Three stacked bars for a 64-unit hidden layer. Task 1 claims 19 units, Task 2 adds 13, and Task 3 adds 8; the unclaimed segment falls from 45 to 32 and then 24 units.
Illustrative HAT mask utilization in a 64-unit hidden layer when three tasks claim 30%, 30%, and 28% of the capacity remaining at their arrival. The unclaimed region shrinks from 45 units after Task 1 to 24 after Task 3, showing how a fixed-capacity mask method gradually exhausts room for later tasks.