Large language models (LMs) such as GPT-3 have the surprising ability to do in-context
learning, where the model learns to do a downstream task simply by conditioning on a prompt …

AI is undergoing a paradigm shift with the rise of models (eg, BERT, DALL-E, GPT-3) that are
trained on broad data at scale and are adaptable to a wide range of downstream tasks. We …

Language models (LMs) are becoming the foundation for almost all major language
technologies, but their capabilities, limitations, and risks are not well understood. We present …

Reward design in reinforcement learning (RL) is challenging since specifying human
notions of desired behavior may be difficult via reward functions or require many expert …

A major factor in the recent success of large language models is the use of enormous and
ever-growing text datasets for unsupervised pre-training. However, naively training a model …

Distribution shifts—where the training distribution differs from the test distribution—can
substantially degrade the accuracy of machine learning (ML) systems deployed in the wild. …

The mixture proportions of pretraining data domains (eg, Wikipedia, books, web text) greatly
affect language model (LM) performance. In this paper, we propose Domain Reweighting …

Selecting a suitable pretraining dataset is crucial for both general-domain (eg, GPT-3) and
domain-specific (eg, Codex) language models (LMs). We formalize this problem as selecting …

While adversarial training can improve robust accuracy (against an adversary), it sometimes
hurts standard accuracy (when there is no adversary). Previous work has studied this …

Adversarial training augments the training set with perturbations to improve the robust error (over
worst-case perturbations), but it often leads to an increase in the standard error (on …