Optimization in ML
Optimization is a key component of machine learning. In this series we review recent developments allowing to train larger models, faster and more reliably.

Research feed

Software

optimagic: unifying the numerical optimization ecosystem in Python

optimagic is a Python package for numerical optimization. It is a unified interface to optimizers from SciPy, NlOpt and other packages. …

Janoś Gabler
Optimization in ML
Pill

Symmetry Teleportation for Accelerated Optimization

A novel approach, symmetry teleportation, enhances convergence speed in gradient-based optimization by allowing parameters to traverse …

Faried Abu Zaid Kristof Schröder
Optimization in ML
Seminar

Second-Order Information and Applications

Kristof Schröder will talk about second-order information and its applications in machine learning.

Kristof Schröder
Optimization in ML
Pill

Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training

The process of pre-training large language models can incur significant expenses. As these models continue to grow in size, there is an …

Kristof Schröder
Optimization in ML
Pill

Learning-rate-free learning by D-Adaptation

Fine-tuning the learning rate (lr) in the training of neural networks is crucial for their performance, and often requires a costly search …

Samuel Burbulla
Optimization in ML

Other series in Advances and fundamentals in ML

Simulation and AI

AI techniques are fundamentally transforming the field of simulation by combining physics-based modeling with data-driven machine learning.

Advances and Fundamentals in ML

Reinforcement Learning

Recent and popular advances in Reinforcement Learning are known to be data-hungry. Attempts to handle this deficit include developing …

Advances and Fundamentals in ML

Diffusion Models

Diffusion models (DM) have become the state of the art for sample quality in generative modelling. They work by sequentially corrupting …

Advances and Fundamentals in ML

Geometric deep learning

Specialized deep learning architectures exploit the intrinsic regularities arising from the underlying structure of the physical world. …

Advances and Fundamentals in ML
Check all of our work