Yoonjoo Lee

I am a Postdoctoral Fellow at the University of Michigan CSE, working with Q. Vera Liao on human-centered AI. I received my Ph.D. in Computer Science from KAIST, where I worked with Juho Kim as a member of KIXLAB. During my Ph.D., I interned at Adobe Research with Alexa F. Siu and Tong Sun, at the Allen Institute for AI with Pao Siangliulue, Joseph Chee Chang, Jonathan Bragg, and Kyle Lo, and at LG AI Research with Moontae Lee.

My research lies at the intersection of human-AI interaction and natural language processing, centered on cognitive alignment: building AI systems that understand, adapt to, and augment how people reason, learn, and work. I pursue this through three complementary directions: (1) developing AI systems that adapt their support to individual users’ tasks, contexts, and evolving needs; (2) designing evaluation frameworks and benchmarks that measure AI capabilities in fine-grained, human-interpretable ways; and (3) building user simulators that enable scalable evaluation and training by modeling how users interact with and respond to AI.

news

Two papers on LLM user simulation accepted to EMNLP 2026 — KnowSim and Assistant Bias in LLM User Simulators!

We’re organizing Revibing HCI: Agentic Reimplementation as Community Practice at UIST 2026 — come join us!

Presented DiscoverLLM at ICML 2026 in Seoul!

Revibing Code from Papers accepted to UIST 2026!

Gave a talk at Adobe Research on Cognitive Alignment: Building AI Systems That Empower Human Thinking.

Gave a talk at Johns Hopkins on Cognitive Alignment: Building AI Systems That Empower Human Thinking.

Evalet received a Best Paper Honorable Mention at CHI 2026! 🎉

Gave a guest lecture in Q. Vera Liao’s EECS 493 class at the University of Michigan.

Mind the Blind Spots was selected as a Senior Area Chair Highlight at EMNLP 2025!

selected publications

EMNLP

KNOWSIM: Evaluating Information Calibration in LLM Assistants with User Simulators that Learn

Yoonjoo Lee, Hyoungwook Jin, Tae Soo Kim, Shaoyang Zhang, Philippe Laban, and Q. Vera Liao

In Proceedings of the Conference on Empirical Methods in Natural Language Processing (Main Conference), 2026

arXiv Bib Website

@inproceedings{lee2026knowsim,
  title = {KNOWSIM: Evaluating Information Calibration in LLM Assistants with User Simulators that Learn},
  author = {Lee, Yoonjoo and Jin, Hyoungwook and Kim, Tae Soo and Zhang, Shaoyang and Laban, Philippe and Liao, Q. Vera},
  booktitle = {Proceedings of the Conference on Empirical Methods in Natural Language Processing (Main Conference)},
  year = {2026},
}

EMNLP

Investigating Assistant Bias in LLM User Simulators Using a Role Vector

Daeheon Jeong, Yoonjoo Lee, Eugene Choi, Sinie van der Ben, and Juho Kim

In Findings of the Conference on Empirical Methods in Natural Language Processing, 2026

Bib

@inproceedings{jeong2026assistantbias,
  title = {Investigating Assistant Bias in LLM User Simulators Using a Role Vector},
  author = {Jeong, Daeheon and Lee, Yoonjoo and Choi, Eugene and {van der Ben}, Sinie and Kim, Juho},
  booktitle = {Findings of the Conference on Empirical Methods in Natural Language Processing},
  year = {2026},
}

UIST

Revibing Code from Papers: Reimplementing HCI Artifacts

Eytan Adar, Yoonjoo Lee, Ning-Er Lei, Q. Vera Liao, and Weirui Peng

In Proceedings of the ACM Symposium on User Interface Software and Technology, 2026

arXiv Bib

@inproceedings{adar2026reviving,
  title = {Revibing Code from Papers: Reimplementing HCI Artifacts},
  author = {Adar, Eytan and Lee, Yoonjoo and Lei, Ning-Er and Liao, Q. Vera and Peng, Weirui},
  booktitle = {Proceedings of the ACM Symposium on User Interface Software and Technology},
  year = {2026},
}

ICML
DiscoverLLM: From Executing Intents to Discovering Them

Tae Soo Kim, Yoonjoo Lee, Jaesang Yu, John Joon Young Chung, and Juho Kim

In Proceedings of the International Conference on Machine Learning, 2026

Abs arXiv Bib

To handle ambiguous and open-ended requests, Large Language Models (LLMs) are increasingly trained to interact with users to surface intents they have not yet expressed (e.g., ask clarification questions). However, users are often ambiguous because they have not yet formed their intents: they must observe and explore outcomes to discover what they want. Simply asking "what kind of tone do you want?" fails when users themselves do not know. We introduce DiscoverLLM, a novel and generalizable framework that trains LLMs to help users form and discover their intents. Central to our approach is a novel user simulator that models cognitive state with a hierarchy of intents that progressively concretize as the model surfaces relevant options – where the degree of concretization serves as a reward signal that models can be trained to optimize. Resulting models learn to collaborate with users by adaptively diverging (i.e., explore options) when intents are unclear, and converging (i.e., refine and implement) when intents concretize. Across proposed interactive benchmarks in creative writing, technical writing, and SVG drawing, DiscoverLLM achieves over 10% higher task performance while reducing conversation length by up to 40%. In a user study with 75 human participants, DiscoverLLM improved conversation satisfaction and efficiency compared to baselines.
@inproceedings{kim2026discoverllm, title = {DiscoverLLM: From Executing Intents to Discovering Them}, author = {Kim, Tae Soo and Lee, Yoonjoo and Yu, Jaesang and Chung, John Joon Young and Kim, Juho}, booktitle = {Proceedings of the International Conference on Machine Learning}, year = {2026}, }
COLM
CUPID: Evaluating Personalized and Contextualized Alignment of LLMs from Interactions

Tae Soo Kim, Yoonjoo Lee, Yoonah Park, Jiho Kim, Young-Ho Kim, and Juho Kim

In Conference on Language Modeling, 2025

Abs arXiv Bib

Personalization of Large Language Models (LLMs) often assumes users hold static preferences that reflect globally in all tasks. In reality, humans hold dynamic preferences that change depending on the context. As users interact with an LLM in various contexts, they naturally reveal their contextual preferences, which a model must infer and apply in future contexts to ensure alignment. To assess this, we introduce CUPID, a benchmark of 756 human-curated interaction session histories between users and LLM-based chat assistants. In each interaction session, the user provides a request in a specific context and expresses their preference through multi-turn feedback. Given a new user request and prior interaction sessions, our benchmark assesses whether LLMs can infer the preference relevant to this request and generate a response that satisfies this preference. With CUPID, we evaluated 10 open and proprietary LLMs, revealing that state-of-the-art LLMs struggle to infer preferences from multi-turn interactions and fail to discern what previous context is relevant to a new request – under 50% precision and 65% recall. Our work highlights the need to advance LLM capabilities for more contextually personalized interactions and proposes CUPID as a resource to drive these improvements.
@inproceedings{kim2025cupid, title = {CUPID: Evaluating Personalized and Contextualized Alignment of LLMs from Interactions}, author = {Kim, Tae Soo and Lee, Yoonjoo and Park, Yoonah and Kim, Jiho and Kim, Young-Ho and Kim, Juho}, booktitle = {Conference on Language Modeling}, year = {2025}, }
EMNLP
ArxivDIGESTables: Synthesizing Scientific Literature into Tables using Language Models

Benjamin Newman^*, Yoonjoo Lee^*, Aakanksha Naik, Pao Siangliulue, Raymond Fok, Juho Kim, Daniel S. Weld, Joseph Chee Chang, and Kyle Lo

In Proceedings of the Conference on Empirical Methods in Natural Language Processing (Main Conference), 2024

Abs arXiv Bib PDF

When conducting literature reviews, scientists often create literature review tables - tables whose rows are publications and whose columns constitute a schema, a set of aspects used to compare and contrast the papers. Can we automatically generate these tables using language models (LMs)? In this work, we introduce a framework that leverages LMs to perform this task by decomposing it into separate schema and value generation steps. To enable experimentation, we address two main challenges: First, we overcome a lack of high-quality datasets to benchmark table generation by curating and releasing arxivDIGESTables, a new dataset of 2,228 literature review tables extracted from ArXiv papers that synthesize a total of 7,542 research papers. Second, to support scalable evaluation of model generations against human-authored reference tables, we develop DecontextEval, an automatic evaluation method that aligns elements of tables with the same underlying aspects despite differing surface forms. Given these tools, we evaluate LMs’ abilities to reconstruct reference tables, finding this task benefits from additional context to ground the generation (e.g. table captions, in-text references). Finally, through a human evaluation study we find that even when LMs fail to fully reconstruct a reference table, their generated novel aspects can still be useful.
@inproceedings{newman2024arxivdigestables, title = {ArxivDIGESTables: Synthesizing Scientific Literature into Tables using Language Models}, author = {Newman, Benjamin and Lee, Yoonjoo and Naik, Aakanksha and Siangliulue, Pao and Fok, Raymond and Kim, Juho and Weld, Daniel S. and Chang, Joseph Chee and Lo, Kyle}, booktitle = {Proceedings of the Conference on Empirical Methods in Natural Language Processing (Main Conference)}, year = {2024}, }
FAccT
One vs. Many: Comprehending Accurate Information from Multiple Erroneous and Inconsistent AI Generations

Yoonjoo Lee, Kihoon Son, Tae Soo Kim, Jisu Kim, John Joon Young Chung, Eytan Adar, and Juho Kim

In Proceedings of the ACM Conference on Fairness, Accountability, and Transparency, 2024

Abs arXiv Bib

As Large Language Models (LLMs) are nondeterministic, the same input can generate different outputs, some of which may be incorrect or hallucinated. If run again, the LLM may correct itself and produce the correct answer. Unfortunately, most LLM-powered systems resort to single results which, correct or not, users accept. Having the LLM produce multiple outputs may help identify disagreements or alternatives. However, it is not obvious how the user will interpret conflicts or inconsistencies. To this end, we investigate how users perceive the AI model and comprehend the generated information when they receive multiple, potentially inconsistent, outputs. Through a preliminary study, we identified five types of output inconsistencies. Based on these categories, we conducted a study (N=252) in which participants were given one or more LLM-generated passages to an information-seeking question. We found that inconsistency within multiple LLM-generated outputs lowered the participants’ perceived AI capacity, while also increasing their comprehension of the given information. Specifically, we observed that this positive effect of inconsistencies was most significant for participants who read two passages, compared to those who read three. Based on these findings, we present design implications that, instead of regarding LLM output inconsistencies as a drawback, we can reveal the potential inconsistencies to transparently indicate the limitations of these models and promote critical LLM usage.
@inproceedings{lee2024onevsmany, title = {One vs. Many: Comprehending Accurate Information from Multiple Erroneous and Inconsistent AI Generations}, author = {Lee, Yoonjoo and Son, Kihoon and Kim, Tae Soo and Kim, Jisu and Chung, John Joon Young and Adar, Eytan and Kim, Juho}, booktitle = {Proceedings of the ACM Conference on Fairness, Accountability, and Transparency}, year = {2024}, }
CHI
PaperWeaver: Enriching Topical Paper Alerts by Contextualizing Recommended Papers with User-collected Papers

Yoonjoo Lee, Hyeonsu B. Kang, Matt Latzke, Juho Kim, Jonathan Bragg, Joseph Chee Chang, and Pao Siangliulue

In Proceedings of the CHI Conference on Human Factors in Computing Systems, 2024

Abs arXiv Bib

With the rapid growth of scholarly archives, researchers subscribe to "paper alert" systems that periodically provide them with recommendations of recently published papers that are similar to previously collected papers. However, researchers sometimes struggle to make sense of nuanced connections between recommended papers and their own research context, as existing systems only present paper titles and abstracts. To help researchers spot these connections, we present PaperWeaver, an enriched paper alerts system that provides contextualized text descriptions of recommended papers based on user-collected papers. PaperWeaver employs a computational method based on Large Language Models (LLMs) to infer users’ research interests from their collected papers, extract context-specific aspects of papers, and compare recommended and collected papers on these aspects. Our user study (N=15) showed that participants using PaperWeaver were able to better understand the relevance of recommended papers and triage them more confidently when compared to a baseline that presented the related work sections from recommended papers.
@inproceedings{lee2024paperweaver, title = {PaperWeaver: Enriching Topical Paper Alerts by Contextualizing Recommended Papers with User-collected Papers}, author = {Lee, Yoonjoo and Kang, Hyeonsu B. and Latzke, Matt and Kim, Juho and Bragg, Jonathan and Chang, Joseph Chee and Siangliulue, Pao}, booktitle = {Proceedings of the CHI Conference on Human Factors in Computing Systems}, year = {2024}, }
CHI
EvalLM: Interactive Evaluation of Large Language Model Prompts on User-Defined Criteria

Tae Soo Kim, Yoonjoo Lee, Jamin Shin, Young-Ho Kim, and Juho Kim

In Proceedings of the CHI Conference on Human Factors in Computing Systems, 2024

Abs arXiv Bib Video

By simply composing prompts, developers can prototype novel generative applications with Large Language Models (LLMs). To refine prototypes into products, however, developers must iteratively revise prompts by evaluating outputs to diagnose weaknesses. Formative interviews (N=8) revealed that developers invest significant effort in manually evaluating outputs as they assess context-specific and subjective criteria. We present EvalLM, an interactive system for iteratively refining prompts by evaluating multiple outputs on user-defined criteria. By describing criteria in natural language, users can employ the system’s LLM-based evaluator to get an overview of where prompts excel or fail, and improve these based on the evaluator’s feedback. A comparative study (N=12) showed that EvalLM, when compared to manual evaluation, helped participants compose more diverse criteria, examine twice as many outputs, and reach satisfactory prompts with 59% fewer revisions. Beyond prompts, our work can be extended to augment model evaluation and alignment in specific application contexts.
@inproceedings{kim2024evallm, title = {EvalLM: Interactive Evaluation of Large Language Model Prompts on User-Defined Criteria}, author = {Kim, Tae Soo and Lee, Yoonjoo and Shin, Jamin and Kim, Young-Ho and Kim, Juho}, booktitle = {Proceedings of the CHI Conference on Human Factors in Computing Systems}, year = {2024}, }

ICML

QASA: Answering Advanced Questions on Scientific Articles

Yoonjoo Lee^*, Kyungjae Lee^*, Sunghyun Park, Dasol Hwang, Jaehyeon Kim, Hong-in Lee, and Moontae Lee

In Proceedings of the International Conference on Machine Learning, 2023

Bib PDF Code

@inproceedings{lee2023qasa,
  title = {QASA: Answering Advanced Questions on Scientific Articles},
  author = {Lee, Yoonjoo and Lee, Kyungjae and Park, Sunghyun and Hwang, Dasol and Kim, Jaehyeon and Lee, Hong-in and Lee, Moontae},
  booktitle = {Proceedings of the International Conference on Machine Learning},
  year = {2023},
}

UIST

Cells, Generators, and Lenses: Design Framework for Object-Oriented Interaction with Large Language Models

Tae Soo Kim, Yoonjoo Lee, Minsuk Chang, and Juho Kim

In Proceedings of the ACM Symposium on User Interface Software and Technology, 2023

Bib PDF Website

@inproceedings{kim2023cells,
  title = {Cells, Generators, and Lenses: Design Framework for Object-Oriented Interaction with Large Language Models},
  author = {Kim, Tae Soo and Lee, Yoonjoo and Chang, Minsuk and Kim, Juho},
  booktitle = {Proceedings of the ACM Symposium on User Interface Software and Technology},
  year = {2023},
}

CHI

DAPIE: Interactive Step-by-Step Explanatory Dialogues to Answer Children’s Why and How Questions

Yoonjoo Lee, Tae Soo Kim, Sungdong Kim, Yohan Yun, and Juho Kim

In Proceedings of the CHI Conference on Human Factors in Computing Systems, 2023

Bib HTML PDF Video Website

@inproceedings{lee2023dapie,
  title = {DAPIE: Interactive Step-by-Step Explanatory Dialogues to Answer Children's Why and How Questions},
  author = {Lee, Yoonjoo and Kim, Tae Soo and Kim, Sungdong and Yun, Yohan and Kim, Juho},
  booktitle = {Proceedings of the CHI Conference on Human Factors in Computing Systems},
  year = {2023},
}

CHI

Promptiverse: Scalable Generation of Scaffolding Prompts Through Human-AI Hybrid Knowledge Graph Annotation

Yoonjoo Lee, John Joon Young Chung, Tae Soo Kim, Jean Y. Song, and Juho Kim

In Proceedings of the CHI Conference on Human Factors in Computing Systems, 2022

Bib HTML PDF Video Website

@inproceedings{lee2022promptiverse,
  title = {Promptiverse: Scalable Generation of Scaffolding Prompts Through Human-AI Hybrid Knowledge Graph Annotation},
  author = {Lee, Yoonjoo and Chung, John Joon Young and Kim, Tae Soo and Song, Jean Y. and Kim, Juho},
  booktitle = {Proceedings of the CHI Conference on Human Factors in Computing Systems},
  year = {2022},
}

CHI

Personalizing Ambience and Illusionary Presence: How People Use “Study with Me” Videos to Create Effective Studying Environments

Yoonjoo Lee, John Joon Young Chung, Jean Y. Song, Minsuk Chang, and Juho Kim

In Proceedings of the CHI Conference on Human Factors in Computing Systems, 2021

Bib HTML PDF Video

@inproceedings{lee2021studywithme,
  title = {Personalizing Ambience and Illusionary Presence: How People Use ``Study with Me'' Videos to Create Effective Studying Environments},
  author = {Lee, Yoonjoo and Chung, John Joon Young and Song, Jean Y. and Chang, Minsuk and Kim, Juho},
  booktitle = {Proceedings of the CHI Conference on Human Factors in Computing Systems},
  year = {2021},
}