Tokens by Michael Brenndoerfer, Week #20

Tokens

GPT-6 Astra Reads and Edits Model Weights by Hand

Today's issue: MIT shows how frontier models can edit raw weights like source code, while gaming rigs squeeze out 290B parameters.

Welcome to #20 of your weekly AI newsletter.

Don't miss the next one 📬

Everything that mattered in AI this week, in five minutes. Occasionally hilarious.

No spam. Unsubscribe anytime.


Smiling cartoon computer

Main Story

Roll Safe Think About It template: Roll Safe taps his head noting that you do not need gradient descent if frontier models can read raw float tensors like source code.
Turns out backprop was just a clumsy approximation of asking an AI to use NumPy.

Weight-Space Programming: When an LLM Performs Brain Surgery on Another Model

Training a neural network has always felt like metallurgy: you pour in data, crank up the thermal noise with gradient descent, and pray the crystalline structure settles into something smart. You never actually touched the weights directly. That was considered unhinged. If you wanted a model to stop hallucinating or fix a logic loop, you gathered more examples and paid an exorbitant power bill.

MIT CSAIL just threw that assumption into a woodchipper. In a new paper on Weight-Space Programming, researchers showed that OpenAI's GPT-6 Astra can inspect raw weight arrays of smaller neural networks and hand-tune their numerical parameters without running a training loop. Given a sandboxed Python shell and a strict 50-step forward-pass budget to test behavior, the model inspects floating-point matrices like text and scripts targeted modifications directly.

In one test, Astra inspected Qwen3.5-0.8B, pinpointed broken reasoning circuits, and edited a microscopic 0.016% of its weights. That surgical adjustment boosted the tiny model's MATH-500 score from 7.9% straight to 48.1%. Astra also deduced which Atari games small networks were playing purely by reading raw weight matrices, identified visual shapes embedded in diffusion layers, and compiled tiny game-playing networks entirely from scratch without training.

Treating trained models as legible software rather than opaque statistical sludge changes the mechanical baseline of model optimization. When frontier intelligence can read the neural circuitry of lesser models, tuning stops looking like biology and starts looking like decompiling a binary.

  • Surgical Precision: Altering just 0.016% of weights jumped Qwen3.5-0.8B math performance from 7.9% to 48.1% with zero backpropagation.
  • Weight Decryption: Astra correctly deduced Atari game targets and diffusion shapes solely by reading static numerical tensors.
  • Tight Test Budget: Edits were planned in a sandboxed NumPy environment with at most 50 forward passes allowed to verify outputs.

Bottom Line: If frontier models can reliably decompile and rewrite weight tensors, model alignment and capability patching will shift from expensive retraining runs to precise, automated neural surgery.

Source: ai-tldr.dev

Don't miss the next one 📬

Everything that mattered in AI this week, in five minutes. Occasionally hilarious.

No spam. Unsubscribe anytime.


Smiling cartoon star

The Feed

  1. Open-source computer agent NorthCUA-Pro demonstrates that 4 million GUI interaction steps and long-horizon RL can beat closed frontier titans. Running on Qwen3.5, the 397B desktop agent humbled closed models like Kimi-K3 across standard benchmarks.

  2. In-context learning has arrived for synthetic biology. Researchers found genome model Evo 2 can generate functional bacteriophage genomes and custom antibodies from just four prompt examples, skipping fine-tuning through genomic chain-of-thought.

  3. Training frontier models on domestic Chinese silicon is getting real. Meituan introduced LongCat-2.5, a 1.6-trillion-parameter multimodal model built on domestic chips with a 1M context window, claiming parity with Claude-Opus-4.8 on agent tasks.

  4. OpenAI is locking down Gulf capital and energy. Sam Altman toured Abu Dhabi to sign deals with ADNOC and G42, deploying frontier models into industrial infrastructure while chasing a slice of a reported $20 billion funding round co-led by UAE funds.

  5. The White House wants public faith in its voluntary safety reviews, but red tape reigns. A FOIA lawsuit forced partial document releases, yet the core seven-page rulebook and the full list of approved Trusted Partners remain shielded state secrets.

  6. Google launched Project Suncatcher with four datacenter TPUs aboard a SpaceX rideshare. Generating 1 kW of power gives it just 20 minutes of battery life per run, which engineers promptly burned to run Gemma 4 and beam down an orbital haiku to Earth.

  7. Anthropic acquired Mithril Medicine. The one-person startup used Claude to coordinate 11 outside wet labs, pushing a rare muscular dystrophy treatment from raw concept to completed mouse trials in under 18 months.

  8. Berkeley researchers created FreeToken to pool consumer GPU, CPU, and RAM into one elastic memory layer. The framework runs 290B mixture-of-experts models on gaming PCs, serving DeepSeek-V4-Flash locally with interactive latency for coding agents.

  9. Autoregression has a fresh challenger in agent coding. DiffuSpace released Clover-35B-A3B-Preview under an Apache-2.0 license, using parallel block diffusion instead of token-by-token generation to reach a 73.6 score on SWE-bench Verified.

  10. SenseTime dropped SenseNova-Loop-9B-Preview, an open multimodal model that executes 24 of its 32 layers twice. This looped architecture squeezes 56 layers of compute from 32 layers of parameter memory, optimizing tool use and document analysis.

  11. Activation classifiers monitor internal hidden states token by token, cutting off malicious outputs and tool calls before dangerous text can finish rendering rather than waiting to police output text after generation.

  12. Toyota affiliate AISIN unveiled a 16nm SoC pairing ARM cores with on-chip STT-MRAM. By housing model weights and the operating system directly on integrated memory, the edge chip boots Tiny YOLOv3 in 65.7 milliseconds, bypassing external DRAM limits.

  13. Deep Principle launched Hydrogen-1.0, a 753B open science model post-trained from GLM-5.3. Targeting chemistry and biology under an MIT license, the team reports a 38.3% pass rate on FrontierScience-Research, edging out closed frontier options.

  14. A Lancet study took Google's AMIE chatbot out of simulation and into primary care. Evaluating 98 real patient interactions, the clinical trial proved that conversational AI can gather histories and prep physician summaries before urgent clinic visits.

  15. Beijing is stepping up scrutiny of frontier labs. China's NDRC announced sweeping regulations governing everything from raw compute to data pipelines, directly targeting domestic champions like DeepSeek, Moonshot AI, and Alibaba with regular audits.

See you next week!

Michael

Smiling cartoon mobile phone

Do you want to be mentioned in the newsletter, or do you have feedback?

Get in Touch
Smiling cartoon computer

Get Tokens in your inbox 📬

Everything that mattered in AI this week, in five minutes. Occasionally hilarious.

Join 5,000+ readers · No spam, unsubscribe anytime.