Leena Mathur

I am a PhD student at Carnegie Mellon University's School of Computer Science in the Language Technologies Institute. My research focuses on the foundations of multimodal AI, social AI, and embodied intelligence in real-world social interaction contexts, with applications to enhance human and societal well-being. At CMU, I am fortunate to be advised by Louis-Philippe Morency. My work is generously supported by the NSF Graduate Research Fellowship.

I completed a BS in Computer Science, BA in Cognitive Science, and BA in Linguistics from the University of Southern California. I am fortunate to have worked with wonderful research mentors — Maja Matarić at the Interaction Lab in USC's Robotics and Autonomous Systems Center, Khalil Iskarous at USC's Linguistics Department, Ralph Adolphs at Caltech, Michael Shindler at UC Irvine, and Rémi Lebret at the École Polytechnique Fédérale de Lausanne (EPFL).

I am grateful for my undergraduate experience learning from USC's AI community and my 4 years leading and growing the engineering school's student AI organization CAIS++, mentored by Bistra Dilkina from USC's Center for AI in Society. I now serve on the CAIS++ alumni advisory board.

Google Scholar  /  Github  /  LinkedIn  /  Twitter

Email: lmathur_at_andrew_dot_cmu_dot_edu

Research     Teaching     Outreach & Service     Honors     Miscellaneous

profile photo profile photo

  • [May 2024]   Starting a research internship at Robust.AI! I will be working on robot perception and human-robot interaction challenges for collaborative mobile robots working alongside and for humans!
  • [Apr 2024]  📝 New position paper on core technical challenges, open questions, and opportunities to advance social intelligence in AI agents (Social-AI)! Our position is anchored in social intelligence concepts and prior decades of Social-AI research across computing communities.
  • [Apr 2024]  Co-organizing the 3rd edition of the Artificial Social Intelligence Workshop at ECCV 2024 in October! We have a great lineup of keynote speakers and group discussions planned in Milan 🇮🇹
  • [Feb 2024]  Co-organizing the Teaching NLP Workshop at ACL 2024! Looking forward to discussing the future of NLP education at ACL in Bangkok 🇹🇭
  • [Jan 2024]  📝 Sotopia was accepted as a spotlight paper to ICLR 2024! SOTOPIA is a framework and environment for studying social intelligence in language agents.
  • [Oct 2023]  📝 Paper accepted at EMNLP! We propose Difference-Masking, a continued pretraining technique for self-supervised learning!
  • [Jul 2023]  📝 Paper accepted at ICMI! We synthesize how affective phenomena have been studied in multimodal interaction, affective computing, and NLP, highlighting understudied areas and future directions. I will present this in October at ICMI in Paris! 🇫🇷

Papers ( * denotes joint first-authors)

Advancing Social Intelligence in AI Agents: Technical Challenges and Open Questions
Leena Mathur, Paul Pu Liang, Louis-Philippe Morency
Pre-Print, Repo
SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents
Xuhui Zhou*, Hao Zhu*, Leena Mathur, Ruohong Zhang, Zhengyang Qi, Haofei Yu, Louis-Philippe Morency, Yonatan Bisk, Daniel Fried, Graham Neubig, Maarten Sap
To Appear in International Conference on Learning Representations (ICLR 2024) [Spotlight]
Paper, Website
Expanding the Role of Affective Phenomena in Multimodal Interaction Research
Leena Mathur, Maja J Matarić, Louis-Philippe Morency
ACM International Conference on Multimodal Interaction (ICMI 2023)
Difference-Masking: Choosing What to Mask in Continued Pretraining
Alex Wilf*, Syeda Akter*, Leena Mathur, Paul Pu Liang, Sheryl Mathew, Mengrou Shou, Eric Nyberg, Louis-Philippe Morency
Findings of Empirical Methods in Natural Language Processing (EMNLP 2023)
Towards Intercultural Affect Recognition: Audio-Visual Affect Recognition in the Wild Across Six Cultures
Leena Mathur, Ralph Adolphs, Maja J Matarić
IEEE International Conference on Automatic Face and Gesture Recognition, Workshop on Learning with Few or Without Annotated Face, Body and Gesture Data (FG 2023) [Oral]
Student misconceptions of dynamic programming: a replication study
Michael Shindler, Natalia Pinpin, Mia Markovic, Frederick Reiber, Jee Hoon Kim, Giles Pierre Nunez Carlos, Mine Dogucu, Mark Hong, Michael Luu, Brian Anderson, Aaron Cote, Matthew Ferland, Palak Jain, Tyler LaBonte, Leena Mathur, Ryan Moreno, Ryan Sakuma
Journal of Computer Science Education (CSE Vol. 32, No. 3, 2022)
Affect-Aware Deep Belief Network Representations for Multimodal Unsupervised Deception Detection
Leena Mathur and Maja J Matarić
IEEE International Conference on Automatic Face and Gesture Recognition (FG 2021) [Long Oral]
Modeling User Empathy Elicited by a Robot Storyteller
Leena Mathur*, Micol Spitale*, Hao Xi, Jieyun Li, Maja J Matarić
International Conference on Affective Computing and Intelligent Interaction (ACII 2021) [Oral]
Speech Representations and Phoneme Classification for Preserving the Endangered Language of Ladin
Zane Durante*, Leena Mathur*, Eric Ye, Sichong Zhao, Tejas Ramdas, Khalil Iskarous
Interspeech, Workshop on ML in Speech and Language Processing (Interspeech 2021)
Unsupervised Audio-Visual Subspace Alignment for High-Stakes Deception Detection
Leena Mathur and Maja J Matarić
IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2021)
Introducing Representations of Facial Affect in Automated Multimodal Deception Detection
Leena Mathur and Maja J Matarić
ACM International Conference on Multimodal Interaction (ICMI 2020) [Oral, Best Paper Award Nomination (1 of 8 papers)] 🏆
Month-Long, In-Home Socially Assistive Robot for Children with Diverse Needs
Caitlyn Clabaugh, Shomik Jain, Balasubramanian Thiagarajan, Zhonghao Shi, Leena Mathur, Kartik Mahajan, Gisele Ragusa, Maja J Matarić
International Symposium on Experimental Robotics (ISER 2018)


  • Research team member in a large-scale CS education study, to inform algorithms curriculum development in universities (2021-2022); Paper published in the Journal of Computer Science Education (CSE Vol. 32, No. 3, 2022)
  • USC Resident Assistant (RA) in residential colleges with ~1000 first-year students, co-taught RA curriculum (2019-2022)
  • Undergraduate Teaching Assistant for USC's Department of Computer Science -- CS 170, Discrete Mathematics (2019)
  • USC Joint Education Project Instructor; designed and taught an 8-week computer science class to third-grade students at 32nd Street School, Los Angeles (2019)

Outreach & Professional Service

Reviewing/Program Committee

  • COLM 2024
  • IROS 2024
  • ACII 2023 (workshop track chair), ACII 2024
  • ICMI 2023
  • International Journal of Social Robotics, 2022
Workshop and Challenge Organization
  • Workshop Organizer: ECCV 2024 Workshop on Artificial Social Intelligence
  • Workshop Organizer: ACL 2024 Workshop on TeachingNLP
  • Challenge Organizer: ICCV 2023 Social-IQ Challenge
  • Workshop Organizer: ICCV 2023 Workshop on Artificial Social Intelligence
  • Workshop Organizer: ACII 2023 Workshop on Social and Affective Intelligence
  • Co-Organized No Joke: We're Talking About LLMs, a seminar and hackathon on large language models (2023)
Mentoring and Outreach
  • Leadership Team running the CMU AI Mentoring Program (2023 - present), mentor undergraduates in this program (2022-present)
  • CMU's PhD and MS Application Info-Session, Language Technologies Institute, School of Computer Science (2022)
  • Volunteer & Panelist at CMU School of Computer Science OurCS Workshop for visiting undergraduate researchers (2022)
  • Worked as a USC Resident Assistant (RA) in residential colleges with ~1000 first-year students, co-taught RA curriculum (2019-2022)
  • Served as a judge at AthenaHacks, an annual hackathon with 400+ participants (2019, 2021, 2022)
  • Mentored undergraduates in USC Women in Engineering Mentorship Program (2019)
  • Mentored undergraduates in the student branch of USC's Center for AI in Society, CAIS++ and USC Interaction Lab (2018-2022)
  • Designed and taught AI and robotics workshops for 500+ USC students and Los Angeles K-12 students; initiated these programs through the Viterbi K-12 STEM Center, CAIS++, and USC Interaction Lab (2018-2022)

Selected Honors

  • National Science Foundation Graduate Research Fellowship (2022-2027)
  • CRA Outstanding Undergraduate Researcher Award [CRA post, USC News] (2022)
  • USC Viterbi School of Engineering Award for Outstanding Research, recognizes one graduating student for research (2022)
  • USC Albert Dorman Grand Challenge Scholar Award, recognizes one graduating student for research, academics, leadership, and service (2022)
  • USC university-wide awards: Discovery Scholar, Global Scholar Prize, Renaissance Scholar, Order of Troy (2022)
  • USC Department of Computer Science Award for Outstanding Research (2022)
  • American Society for Engineering Education PSW Undergraduate Student Award (2022)
  • Astronaut Scholarship (2021)
  • Caltech Summer Undergraduate Research Fellowship (2021)
  • USC Robotics Lab George Bekey Service Award, for contributions and mentorship in the USC robotics community (2021)
  • Phi Beta Kappa (2021)
  • Barry M. Goldwater Scholarship [USC News] (2020)
  • Best Paper Award Nominee at ACM ICMI (2020)
  • USC Provost's Undergraduate Research Fellowship (7 times, 2019-2021)
  • École Polytechnique Fédérale de Lausanne (EPFL) Summer Computer Science Research Fellowship (2019)


Some resources regarding academia and research that I have found helpful:

  • Paul Liang's repo which includes many of the highlighted items on this list (and more)
  • Eugene Vinitsky's post on expectation setting during a PhD
  • Pranav Rajpurkar's materials on AI research experiences
  • Mor Harchol-Balter's description of the CS PhD application process
  • Andrej Karpathy's survival guide for CS PhD students
  • Tim Dettmers' guide to CS PhD admissions (with a focus on AI)
  • Another golden guide from Tim Dettmers on how to choose a CS PhD program
  • Alex Lang's resources on applying for the NSF GRFP
  • Mallory Ladd's resources on applying for the NSF GRFP
  • Andrej Karpathy's perspective on navigating undergraduate courses

I learned piano for ~13 years growing up and continue to enjoy piano. My favorite composers are Chopin, Brahms, Liszt, and Rachmaninoff.

last updated on May 11, 2024
created by modifying this wonderful template from Jon Barron.