Forcing LLMs to be evil during training can make them nicer in the long run

Forcing LLMs to be evil during training can make them nicer in the long run

New Anthropic research shows that undesirable LLM traits can be detected—and even prevented—by examining and manipulating the model’s inner workings.

Forcing LLMs to be evil during training can make them nicer in the long run – Mnnofa

For this study, Lindsey and his colleagues worked to lay down some of that groundwork. Previous research has shown that various dimensions of LLMs’ behavior—from whether they are talking about weddings to persistent traits such as sycophancy—are associated with specific patterns of activity in the simulated neurons that constitute LLMs. Those patterns can be written down as a long string of numbers, in which each number represents how active a s…