What did 4-bit lose?
The average 4-bit cost is tiny: +0.0018 loss, 5% of the adapter's gain. The average hides the tail. These are the rows where the 4.88 MB adapter moves furthest from the 29.6 MB fp32 one.
Largest 4-bit regressions
Where 4-bit helped
Quantization is not monotone: on some rows the int8/4-bit weights land closer to the original than the fp32 ones do. These are the five largest improvements.
How to read this
Loss is masked SFT loss on a single Dolly row. Lower is better. The 8-bit column is usually indistinguishable; the 4-bit column is the one with a tail. A second disjoint seed did not replicate the +0.0018 average, so this is a diagnostic of where a 4-bit adapter can move, not a fixed tax.
If you ship a 4-bit adapter, test the tail: factual lookups, lists and short answers are where the largest deltas appear here.