Research feed: Large Language Models

Staying abreast in the fast-paced world of machine learning research is hard. We contribute to the community by writing paper pills: summaries of publications, talks or new software, with context and an explanation of the main ideas and key results.

Paper pills are written for practitioners or researchers who want a peek into recent developments but are short on time. We publish monthly digests as well as occasional highlights of major conferences.

CLP-Transfer: Cross-Lingual and Progressive Transfer Learning

The CLP-Transfer method introduces a novel approach for cross-lingual language transfer by leveraging token overlap and a small pre-trained …

Anes Benmerzoug
Large Language Models

Position: Leverage Foundational Models for Black-Box Optimization

This paper explores the use of Large Language Models (LLMs) to address challenges in Black Box Optimization (BBO), particularly …

Jan Teusen
Large Language Models

WECHSEL: Cross-Lingual Transfer

Language transfer enables the use of language models trained in one or more languages to initialize a new language model in another …

Anes Benmerzoug
Large Language Models

Reasoning Traces as Learning Signal

An important feature of large language models is their ability to provide detailed responses that resemble “thinking step by …

Robert Müller
Large Language Models

Direct Preference Optimization

With direct preference optimization (DPO), a language model can be aligned with human preferences without using reinforcement learning, …

Michael Panchenko
Large Language Models

Augmented Language Models: a survey

A survey of recent advances in augmenting (large) language models with new capabilities such as reasoning, tool use, and more. While the …

Michael Panchenko
Large Language Models

DictBERT: Dictionary Description Knowledge Enhanced Language Model Pre-training via Contrastive Learning

External knowledge can be injected into pre-trained language models (here specifically BERT) by training an additional language model on …

Michael Panchenko
Large Language Models