An Introduction to Normalizing Flows

Normalizing Flows are a new class of methods that transform a simple input distribution (e.g. Gaussian) into a complex distribution through a series of invertible mappings.

This is part 1 on Normalizing Flows.

Introduction to Normalizing Flows

We consider the topic of modeling probability distributions given samples of that unknown distribution (generative modeling)
Several methods in generative modeling exist. Popular, recent approaches are GANs and VAEs.
Normalizing Flows are a new class of methods that transform a simple input distribution (e.g. Gaussian) into a complex distribution through a series of invertible mappings.
Many types of such mappings have been proposed, e.g. linear, or 1x1 convolutions.
Normalizing flows combine many advantageous features which a generative modeling technique should have: No restriction to a predetermined class of density functions, direct calculation of prob. densities in any point, trainable network parameters.

Demo: Coupling Normalizing Flows in Python

References

[Kob20N]

Normalizing Flows: An Introduction and Review of Current Methods, Ivan Kobyzev, Simon J. D. Prince, Marcus A. Brubaker.

Apr 2020

Normalizing Flows are generative models which produce tractable distributions where both sampling and density evaluation can be efficient and exact. The goal of this survey article is to give a coherent and comprehensive review of the literature around the construction and use of Normalizing Flows for distribution learning. We aim to provide context and explanation of the models, review current …

[Pap21N]

Normalizing flows for probabilistic modeling and inference, George Papamakarios, Eric Nalisnick, Danilo Jimenez Rezende, Shakir Mohamed, Balaji Lakshminarayanan.

Jan 2021

Normalizing flows provide a general mechanism for defining expressive probability distributions, only requiring the specification of a (usually simple) base distribution and a series of bijective transformations. There has been much recent work on normalizing flows, ranging from improving their expressive power to expanding their application. We believe the field has now matured and is in need of …

References

In this series →