Karina Nguyen

I lead a research team at OpenAI, creating new interaction paradigms for reasoning interfaces and capabilities like ChatGPT canvas, streaming chain-of-thought for o1 models, and more via novel synthetic model training.

Previously, I worked at Anthropic:

  • Post-training (RLAIF/RLHF) and leading evaluations for the Claude 3 model series
  • Training & productionizing cost-performance Pareto frontier models: Claude 3 Haiku, Claude Instant 1.2 in the API
  • Researching Constitutional AI, model behavior, hallucinations, honesty, self-correction, refusals, faithful reasoning
  • Building the file uploads feature to productionize 100K long-context capability
  • Developing long-horizon & human feedback interfaces, Claude in Slack, and other unreleased products
  • Writing the first 50,000 lines for the claude.ai and developer console
  • +100 more things that you’d expect to do in a very fast-growing startup!

Previously, as a product engineer/designer, I worked on R&D prototypes, engineering tools, and product features with teams at Primer.ai, Dropbox, Square, and the New York Times.

Research

Find all published research here.

SimpleQA: Measuring Short-form Factuality in Large Language Models

Jason Wei*, Nguyen Karina*, Hyung Won Chung, Yunxin Joy Jiao, Spencer Papay, Amelia Glaese, John Schulman, William Fedus

A factuality benchmark called SimpleQA that measures the ability for language models to answer short, fact-seeking questions.

Question Decomposition Improves the Faithfulness of Model-Generated Reasoning

Ansh Radhakrishnan*, Karina Nguyen*, +18 more, Jared Kaplan, Jan Brauner, Samuel R. Bowman, Ethan Perez

By forcing the model to answer simpler subquestions in separate contexts, we greatly increase the faithfulness of model-generated reasoning over chain-of-thought (CoT), while still achieving some of the performance gains of CoT.

Towards Measuring the Representation of Subjective Global Opinions in Language Models

Esin Durmus, Karina Nguyen, Thomas I Liao, Nicholas Schiefer, +11 more, Jared Kaplan, Jack Clark, Deep Ganguli

We develop a method to test global opinions represented in language models.

In submission 2023

Discovering Language Model Behaviors with Model-Written Evaluations

Ethan Perez, Sam Ringer*, Kamile Lukošiute*, Karina Nguyen*, Edwin Chen, Scott Heiner, +55 more, Nicholas Schiefer, Jared Kaplan

We test LMs using >150 LM-written evaluations, finding cases of inverse scaling, where models exhibit sycophantic behaviors.

ACL'23 (Findings)

Towards Semantically-Aware UI Design Tools: Design, Implementation and Evaluation of Semantic Grouping Guidelines

Peitong Duan, Bjorn Hartmann, Karina Nguyen, Yang Li, Marti Hearst, Meredith Ringel Morris

We develop computational metric to measure semantic grouping UI violations.

ICML Workshop'23

Investigations

My work in visual forensics & human rights contributed to the Pulitzer Prize reporting. That involved investigations of war crimes and crimes against humanity with extensive data collection, evidence verification, satellite analysis, 3D reconstructions, legal submissions, investigative tools, and applied remote sensing.

Say Hi!