Towards AI Understanding Humans

I build computational models that enable AI to understand people in interactive tasks. It allows for the simulation of realistic human interactions and deepens AI’s understanding of how people interact with environments. This relies not only on humans' goals, knowledge, and experience, but also on the complex inner mechanisms of perception, cognition, and motor control.

Publications:

[1] Typoist: Simulating Errors in Touchscreen Typing [website]

[2] Chartist: Task-driven Eye Movement Control for Chart Reading [website]

[3] CRTypist: Simulating Touchscreen Typing Behavior via Computational Rationality [website]

[4] A Workflow for building Computationally Rational Models of Human Behavior [paper]

[5] WigglyEyes: Inferring Eye Movements from Keypress Data [paper]

---

Towards AI Integrating Human Feedback

I build interactive systems to effectively represent and analyze agent behaviors that help effectively integrate human feedback. By making complex behavioral data more accessible and interpretable, they facilitate the effective alignment of AI behaviors with human feedback.

Publications:

[1] DxHF: Providing High-Quality Human Feedback for LLM Alignment with Interactive Decomposition [website]

[2] Interactive Reward Tuning: Interactive Visualization for Preference Elicitation [website]

[3] Interactive Groupwise Comparison for Reinforcement Learning from Human Feedback [website]

---

Visualization Tools

Calliope (project homepage) is a visual data story generation platform that employs advanced AI techniques to automatically analyze data and represent data insights in the form of narrative visualization.

Narrative Chart (narchart.github.io) is an open-source visualization library for authoring charts that facilitete data storytelling with a high-level action-orienented declarative grammar.

Publications:

[1] Calliope: Automatic Visual Data Story Generation from a Spreadsheet [website]

[2] AutoClips: An Automatic Approach to Video Generation from Data Facts [website]

[3] Calliope·Data: An Intelligent Visual Data Story Generation Platform for Mass [website]

[4] Understanding and Automating Graphical Annotations on Animated Scatterplots [paper]

[5] VisAct: a visualization design system based on semantic actions [website]

[6] Task-Oriented Optimal Sequencing of Visualization Charts [website]

[7] Talk2Data: A Natural Language Interface for Exploratory Visual Analysis via Question Decomposition [paper]