Preprint proposes sparse-frame method for transformer fine-tuning

A preprint introduces FrameFT, a parameter-efficient fine-tuning method that represents model updates with sparse coefficients in a Fusion Frame basis shared across layers. The authors say this reduces the memory required to store and optimize updates, and report tests on supervised language tasks plus an application to vision models. Their experiments claim performance comparable to or better than existing parameter-efficient fine-tuning techniques while using fewer trainable parameters.