Preprint reports training gains from adjusting domain loss weights

Researchers report that adjusting how strongly different training domains contribute to learning improved all five evaluated benchmarks for a 1.5B language model trained on 240B tokens. Their preprint reports an average gain of +7.84 percentage points over a matched direct mixture, including +17.06 on GSM8K and +9.00 on MATH. The proposed LARE method automatically adjusts domain weights using running loss statistics while keeping the sampled data mixture unchanged. It changes the influence of training examples without adding domain exposure or requiring additional runs to search for weights. The reported benchmark results concern the evaluated 1.5B model.