2026/07/09 by Zekai Shang
Computer Science · #cs.LG
Direct low-precision write-back can erase optimizer proposals, while aggregate update visibility need not identify parameters worth protecting. We study two uses of high-precision reference traces: candidate-matched auditing before a low-precision run and regime-matched allocation before unseen targets. The exact target-code event is distinct from aggregate proxies, which are not pathwise certificates. In a controlled two-layer grid, an archived proxy tracks crossings spanning 384x in time, with median predicted/measured ratio 1.000 and 94.5% within 15%. In analytic-grid GPT-2-124M/OpenWebText runs, changing only post-optimizer write-back from round-to-nearest to stochastic rounding recovers 89.2-92.2% of the loss gap through 10,000 steps. The effect repeats in a separate modern-decoder bundle, but its visibility trajectories are uninformative. In a fully prospective 162.2M-parameter allocation test, a 600-step source trace freezes a 10% fp32-master tile mask before three unseen targets. It recovers 60.4-61.0% (mean 60.7%), versus 60.5-61.5% (mean 60.8%) for target-specific masks and 17.5-18.2% for the best fixed composition-matched controls; freeze-only recovers 0.7-0.9%. A separately frozen 197.6M-parameter depth migration recovers 65.9-67.2% (mean 66.4%). A matched systems check reduces PyTorch peak allocation by 584.65 MB versus a blanket master, but adds 65.85 MB and has 7.196% lower arm-median timed-window throughput versus direct round-to-nearest, with no validated break-even. A post hoc same-grid ECO-Adam comparator recovers 96.4-96.6%, versus 60.4-61.0% for the source mask; its systems cost is unmeasured. Thus reference traces can audit aggregate visibility and separately guide exact-budget protection, but the audit did not transfer diagnostically and the current allocator has no net systems benefit.