Work Trail

Two memory-cached linear attention variants training on 25k FMA tracks. Early results: the hybrid nearly matches full MC-LA on loss while running 1.9x faster. GRM gating is learning selectivity. FMA-L

Applying Memory Caching (MC) from arXiv:2602.24281 to Mamba and linear attention for music generation — a novel extension the original paper never tested. Architecture decisions, Output Activation Cac

Baseline comparison of Transformer vs Hybrid Mamba-Attention (1:3) for autoregressive music generation over DAC tokens. The hybrid matches the transformer on all codebook metrics while being architect

The sign mismatch was an optimization artifact. The magnitude gap is not. Here's what I think explains it and how I'm testing that.

Phase-2 update: the low-k mid-layer proxy mismatch weakens and reverses with larger SAE training budgets, implying a regime-dependent, optimization-heavy story in the tested setting.

Posted scaffold archive for the paper draft workflow, including main.tex, references, and build instructions.

Working manuscript structure for the SAE proxy-gap program: claim ladder, pre-registered decision rules, and experiment plan.

Visual concept map and decision tree for activation-space vs behavior-space metrics, including calibrated status labels and test gates.

Initial published writeup on the depth-localized SAE proxy gap: high R^2 can still coincide with degraded CE preservation.