Studying LLMs with Influence Functions

Fabio Peruzzo, Senior AI Engineer at appliedAI Initiative, will talk about a recent work on the use of influence functions to study the behavior of LLMs.


In this seminar we explore influence functions, a tool designed to quantify the impact of individual training samples on a model’s predictions. Our discussion will focus on a recent study that employs an approximation method, known as EK-FAC, to significantly reduce the computational cost of obtaining reliable influence scores. We will outline how this approach facilitates the examination of learning patterns in a large-scale language model with 52 billion parameters, demonstrating the potential of influence functions in enhancing our understanding of neural network behavior.


In this series