Researchers find that retraining only small parts of AI models can cut costs and prevent forgetting

Researchers find that retraining only small parts of AI models can cut costs and prevent forgetting

Enterprises often find that when they fine-tune models, one effective approach to making a large language model (LLM) fit for purpose and grounded in data is to have the model lose some of its abilities. After fine-tuning, some models “forget” how to perform certain tasks or other tasks they already learned. Research from the University of Illinois Urbana-Champaign proposes a new method for retraining models that avoids “catastrophic forgetting,…