Studying LLMs with Influence Functions

29 Feb 24 15:00 UTC

Attributions of value to training samples can be used to examine data, improve data acquisition, debug and improve models or compensate data providers. Recent developments in the field enable principled and useful definitions of value which …

Fabio Peruzzo (appliedAI Initiative)

After receiving his PhD in Applied Mathematics with a specialization in complex systems, Fabio was drawn to the potential of the burgeoning field of machine learning. This interest led him to transition into AI engineering. His professional journey includes roles at several startups in the UK. As of 2021, he has been contributing his expertise to the appliedAI Transferlab team. His primary areas of research encompass data valuation, interpretable AI, and reinforcement learning.

Fabio Peruzzo, Senior AI Engineer at appliedAI Initiative, will talk about a recent work on the use of influence functions to study the behavior of LLMs.

Abstract

In this seminar we explore influence functions, a tool designed to quantify the impact of individual training samples on a model’s predictions. Our discussion will focus on a recent study that employs an approximation method, known as EK-FAC, to significantly reduce the computational cost of obtaining reliable influence scores. We will outline how this approach facilitates the examination of learning patterns in a large-scale language model with 52 billion parameters, demonstrating the potential of influence functions in enhancing our understanding of neural network behavior.

References

[Ham05R]

Robust Statistics: The Approach Based on Influence Functions, Frank R. Hampel, Elvezio M. Ronchetti, Peter J. Rousseeuw, Werner A. Stahel.

Apr 2005

Introducing concepts, theory, and applications, Robust Statistics is accessible to a broad audience, avoiding allusions to high-powered mathematics while emphasizing ideas, heuristics, and background. The text covers the approach based on the influence function (the effect of an outlier on an estimater, for example) and related notions such as the breakdown point. It also treats the …

[Tra22P]

pyDVL: The Python Data Valuation Library, Team TransferLab.

2022

pyDVL collects algorithms for Data Valuation and Influence Function computation.

[Koh17U]

Understanding Black-box Predictions via Influence Functions, Pang Wei Koh, Percy Liang.

Jul 2017

How can we explain the predictions of a black-box model? In this paper, we use influence functions — a classic technique from robust statistics — to trace a model’s prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. To scale up influence functions to modern machine learning settings, we develop a …

[Sch22S]

Scaling Up Influence Functions, Andrea Schioppa, Polina Zablotskaia, David Vilar, Artem Sokolov.

Jun 2022

We address efficient calculation of influence functions for tracking predictions back to the training data. We propose and analyze a new approach to speeding up the inverse Hessian calculation based on Arnoldi iteration. With this improvement, we achieve, to the best of our knowledge, the first successful implementation of influence functions that scales to full-size (language and vision) …

[Geo18F]

Fast Approximate Natural Gradient Descent in a Kronecker Factored Eigenbasis, Thomas George, César Laurent, Xavier Bouthillier, Nicolas Ballas, Pascal Vincent.

2018

Optimization algorithms that leverage gradient covariance information, such as variants of natural gradient descent (Amari, 1998), offer the prospect of yielding more effective descent directions. For models with many parameters, the covari- ance matrix they are based on becomes gigantic, making them inapplicable in their original form. This has motivated research into both simple diagonal …

[Gro23S]

Studying Large Language Model Generalization with Influence Functions, Roger Grosse, Juhan Bae, Cem Anil, Nelson Elhage, Alex Tamkin, Amirhossein Tajdini, Benoit Steiner, Dustin Li, Esin Durmus, Ethan Perez, Evan Hubinger, Kamilė Lukošiūtė, Karina Nguyen, Nicholas Joseph, Sam McCandlish, Jared Kaplan, Samuel R. Bowman.

Aug 2023

When trying to gain better visibility into a machine learning model in order to understand and mitigate the associated risks, a potentially valuable source of evidence is: which training examples most contribute to a given behavior? Influence functions aim to answer a counterfactual: how would the model's parameters (and hence its outputs) change if a given sequence were added to the training set? …

[Mar15O]

Optimizing Neural Networks with Kronecker-factored Approximate Curvature, James Martens, Roger Grosse.

Jun 2015

We propose an efficient method for approximating natural gradient descent in neural networks which we call Kronecker-factored Approximate Curvature (K-FAC). K-FAC is based on an efficiently invertible approximation of a neural network’s Fisher information matrix which is neither diagonal nor low-rank, and in some cases is completely non-sparse. It is derived by approximating various large blocks …

[Pru20E]

Estimating Training Data Influence by Tracing Gradient Descent, Garima Pruthi, Frederick Liu, Mukund Sundararajan, Satyen Kale.

Feb 2020

We introduce a method called TracIn that computes the influence of a training example on a prediction made by the model. The idea is to trace how the loss on the test point changes during the training process whenever the training example of interest was utilized. We provide a scalable implementation of TracIn via: (a) a first-order gradient approximation to the exact computation, (b) saved …

[Fis23I]

Influence Diagnostics under Self-concordance, Jillian Fisher, Lang Liu, Krishna Pillutla, Yejin Choi, Zaid Harchaoui.