Studying LLMs with Influence Functions

Fabio Peruzzo, Senior AI Engineer at appliedAI Initiative, will talk about a recent work on the use of influence functions to study the behavior of LLMs.

Abstract

This seminar we delve into the realm of influence functions, a powerful tool designed to quantify the impact of individual training samples on a model’s predictions. Our discussion will focus on a recent breakthrough study that employs an approximation method, known as EK-FAC, to significantly reduce the computational cost of obtaining reliable influence scores. We will outline how this approach facilitates the examination of learning patterns in a large-scale language model with 52 billion parameters, demonstrating the potential of influence functions in enhancing our understanding of neural network behavior.

References

In this series