AI Safety

A model that scores well in evaluation can still fail once it is deployed, and the failures that matter are rarely the ones a benchmark measures. We study how vision and language systems behave under distribution shift, how their failure modes can be found and written down before users run into them, and what happens when a system is given the chance to learn from its own past mistakes.

Some of this is robustness in the ordinary sense, where the question is how far a model can be pushed off its training distribution before it stops being reliable. Some of it concerns agents, where errors compound over long runs and one bad step can be repeated indefinitely if nothing catches it. The aim in both cases is to know the limits of a system before anyone has to depend on it.