Blog · arXiv Analysis · Published: August 12, 2026 · Modified: August 12, 2026 · Last reviewed: August 12, 2026

The Compressed Model Becomes the Language Tax

Compression can distribute quality loss unevenly across languages, even when every user begins from the same uncompressed checkpoint.

A small post-hoc repair narrows that gap in one controlled setting, but its uneven results make the audit trail more important than the headline.

The Paper

The source is Nirmal Thomas's Language-Conditional Dequantization: Recovering What Quantization Steals from Non-English Languages, arXiv:2608.11786v1 [cs.CL], submitted August 12, 2026. The paper lists Prathama International as the author's affiliation. It studies a narrow but consequential deployment question: after a multilingual model is compressed using English calibration data, can a small language-specific correction recover some of the lost performance?

Here, “language tax” is a governance metaphor for an additional measured quality loss under one compression configuration. It is not a price, a statement about a user's identity, or evidence that every quantization method produces the same disparity. The useful object of review is the complete artifact path: full-precision checkpoint, calibration corpus, quantizer, compressed checkpoint, language, metric, correction, and downstream task.

Calibration Is a Design Choice

Post-training quantization reduces the precision used to store or compute with model weights. Its calibration examples help the quantizer approximate the behavior it should preserve. In the paper's experimental setup, both Qwen2.5-3B and Llama-3.2-3B were converted with GPTQ W3A16: 3-bit weights, 16-bit activations, group size 128, and 128 English C4 calibration samples. English was therefore not merely another evaluation language; it shaped the compression procedure.

The measured perplexity ratios expose the consequence within that setup. On Qwen2.5-3B, INT3 perplexity was 1.35 times the FP16 value for English, compared with 4.37 times for Arabic, 3.49 for Korean, and 3.04 for Japanese. Llama-3.2-3B showed the same ordering at different magnitudes. These are model-and-procedure results, not a universal ranking of languages or their speakers.

The Experiment

Thomas proposes Language-Conditional Dequantization, or LCD: rank-2 LoRA-style corrections attached to the linear layers of an already-quantized model. According to the method section, each language correction adds 0.12 percent parameters, trains only those correction parameters on 256 monolingual C4 or mC4 samples for 500 steps, and takes about 15 to 20 minutes on one L4 GPU. Each resulting delta is approximately 7 MB. At inference, a locale, explicit selection, or language detector chooses the correction slot.

The evaluation covered English plus Arabic, Japanese, Chinese, Hindi, Russian, French, Spanish, and Korean. Perplexity used 32 held-out C4 or mC4 samples per language; GlobalMMLU accuracy used log-likelihood over approximately 14,040 items and 57 subjects per language. This separation matters because a repair can improve next-token probability without restoring the reasoning behavior a task benchmark asks for.

Recovery Is Metric-Dependent

LCD recovered between about 70 and 83 percent of the perplexity gap for Arabic, Japanese, Chinese, and Korean, while the non-English averages were 63.5 percent for Qwen and 69.1 percent for Llama. Yet the GlobalMMLU result was much smaller: average gap recovery was 28 percent for Qwen and 17 percent for Llama. Llama therefore showed the cleaner perplexity repair and the weaker downstream recovery.

The paper's diagnosis attributes that disconnect to where activation error appeared: later layers in Qwen and earlier-middle layers in Llama. That mechanism is the author's interpretation of controlled layer-error measurements, not a general law of model architecture. The governance conclusion needs less certainty: perplexity is not a substitute for task evaluation. A team that reports only the larger recovery percentage would describe a different product from the one users encounter.

Conditional Is Not Universally Better

The most important comparison limits the method's name. On four Qwen languages, per-language LCD recovered an average 65.4 percent of the perplexity gap; a same-rank, same-budget language-agnostic adapter recovered 64.6 percent. The per-language table shows LCD ahead by 8.9 points for Arabic, 7.0 for Japanese, and 3.0 for Korean, but behind by 15.8 for French. Conditioning helped where the tested data contained a distinct correction signal; it was not a universal advantage.

This counter-result changes deployment policy. “One adapter per language” should not become an automatic procurement rule. A deployment should compare language-specific, shared multilingual, and no-correction baselines for each language-task pair, then select the smallest intervention that passes its acceptance test.

The Scale-Up Refuses the Headline

The paper's limitations add a Qwen2.5-7B probe. Rank-2 LCD recovered only 8.5 percent of the non-English GlobalMMLU gap, compared with 28 percent for the 3B model. Increasing the adapter to rank 4 and training for 1,000 steps made every evaluated language worse, with an average reported recovery of negative 3.9 percent; the author interprets this as overfitting the 256-sample training slice. More model and adapter capacity did not preserve the 3B result.

The Claim Boundary

The paper's claim boundary covers two base models below 4B parameters, one English-calibrated INT3 GPTQ recipe, and eight relatively high-resource non-English languages. It does not establish the result for AWQ, GGUF k-quants, SmoothQuant, other bit widths, multilingual calibration, mixture-of-experts systems, or instruction-tuned variants. It does not test endangered or very-low-resource languages, dialects, code-switched prompts, legal or medical domains, toxicity, factuality, or stereotypes. Correct adapter selection also requires a known language; mixed and misidentified inputs remain uncharacterized.

The arXiv source archive contains manuscript source, bibliography, and figure assets, but not a reproducible experiment package with code, data manifests, and configuration files. The numerical findings can be inspected in the paper; independent execution cannot be inferred from that archive alone.

The Compression Receipt

A compression receipt should preserve the base checkpoint and hash, tokenizer, quantizer and version, weight and activation precision, group size, calibration corpus and language mix, sample selection, hardware, compressed-artifact hash, evaluation languages, dialect and code-switch policy, per-language perplexity and task results, correction type and rank, training data and steps, language-routing method, error cases, uncorrected and shared-adapter baselines, reviewer, acceptance threshold, rollback rule, and date. Aggregate averages should never replace the per-language table.

The Governance Standard

Do not certify a compressed endpoint as equivalent to its base model from storage savings, English results, or perplexity alone. Test the deployed quantization recipe across the languages and tasks the service actually claims to support. Keep negative comparisons visible, treat adapter routing as part of the product, and re-run the receipt when the checkpoint, calibration mix, quantizer, precision, adapter, or task distribution changes. Compression is a model change, and its burden can have a language.

Sources

Return to Blog