Research

Voice Agents Evaluations & Full-Duplex Speech Language Modeling

• APEX-Voice-Healthcare: Benchmarking Voice Agents in Professional Healthcare Workflows Puneet Mathur, Dinesh Manocha In submission, 2026 · (Coming soon)
• MultiPartyTurnBench: A Benchmark for Multi-Party End-of-Turn and Interruption Detection Puneet Mathur, Dinesh Manocha [Project Page] In submission, 2026
• APEX-Voice: Can Voice Agents Complete Professional Workflows through Full-Duplex Interaction Puneet Mathur, Dinesh Manocha [Project Page] In submission, 2026
• FD-VAD: Semantic Endpoint Detection for Streaming Full-Duplex Speech Puneet Mathur, Dinesh Manocha [Project Page] In submission, 2026
• Duplex-R1: Full-Duplex Audio LMs that Reason, Retrieve, and Speak While Searching Puneet Mathur, Nedim Lipka, Zeyu Jin, Dinesh Manocha In submission, 2026 · (Coming soon)
• DuplexSpeechBench–IFEval: Evaluating Implicit Instruction Following in Full-Duplex Voice Agents Puneet Mathur, Dinesh Manocha [Project Page] In submission, 2026
• DuplexSpeechBench–Document Grounding: Benchmarking Document Grounding and Hallucinations in Full-Duplex Voice Agents Puneet Mathur, Nedim Lipka, Zeyu Jin, Dinesh Manocha [Project Page] In submission, 2026
• Omni2LoRA: Coherence-Preserving Parametric Memory for Efficient Omni Language Models Puneet Mathur, Manan Suri, Dinesh Manocha [Project Page] In submission, 2026

Agentic Evals & Benchmarking

• AnalystBench: Benchmarking Professional Long-Form Report Generation with Web-Mined Multimodal Tasks Chau Minh Pham, Zichao Wang, Puneet Mathur, Alexa Siu, Akriti Jain, Aparna Garimella, Ananya B Sai, Nedim Lipka, Mohit Iyyer, Varun Manjunatha ACL 2026
• DRAGON: A Benchmark for Evidence-Grounded Visual Reasoning over Diagrams Anirudh Iyengar Kaniyar Narayana Iyengar, Tampu Ravi Kumar, Gaurav Najpande, Manan Suri, Dinesh Manocha, Puneet Mathur, Vivek Gupta NeurIPS 2026
• ChartEval: LLM-Driven Chart Generation Evaluation Using Scene Graph Parsing Kanika Goswami, Puneet Mathur, Ryan A. Rossi, Franck Dernoncourt, Vivek Gupta, Dinesh Manocha AACL 2026

Agentic AI, LLM Reasoning & Reinforcement Learning

• ARGUS: Structured Uncertainty guided Clarification for LLM Agents Manan Suri, Puneet Mathur, Nedim Lipka, Franck Dernoncourt, Ryan A. Rossi, Dinesh Manocha ACL 2026
• Partial Policy Gradients for RL in LLMs Puneet Mathur, Branislav Kveton, Subhojyoti Mukherjee, Viet Dac Lai In submission, 2026
• Cluster-R1: Large Reasoning Models Are Instruction-following Clustering Agents Peijun Qing, Puneet Mathur, Nedim Lipka, Varun Manjunatha, Ryan A. Rossi, Franck Dernoncourt, Saeed Hassanpour, Soroush Vosoughi In submission, 2026

Multimodal Document Intelligence, Attribution & RAG

• DRAGON: A Benchmark for Evidence-Grounded Visual Reasoning over Diagrams Anirudh Iyengar Kaniyar Narayana Iyengar*, Tampu Ravi Kumar*, Gaurav Najpande, Manan Suri, Dinesh Manocha, Puneet Mathur, Vivek Gupta Neurips 2026
• ChartEval: LLM-Driven Chart Generation Evaluation Using Scene Graph Parsing Kanika Goswami, Puneet Mathur, Franck Dernoncourt, Ryan A. Rossi, Vivek Gupta, Dinesh Manocha AACL 2025
• ChartLens: Fine-grained Visual Attribution in Charts Manan Suri, Puneet Mathur, Nedim Lipka, Franck Dernoncourt, Ryan A. Rossi, Dinesh Manocha ACL 2025
• Follow the Flow: Fine-grained Flowchart Attribution with Neurosymbolic Agents Manan Suri, Puneet Mathur, Nedim Lipka, Franck Dernoncourt, Ryan A. Rossi, Vivek Gupta, Dinesh Manocha EMNLP 2025
• DELOC: Document Element Localizer Hammad Ayyubi, Puneet Mathur, Mehrab Tanjim, Vlad I Morariu EMNLP 2025
• SQLSpace: A Representation Space for Text-to-SQL to Discover and Mitigate Robustness Gaps Neha Srikanth, Victor Bursztyn, Puneet Mathur, Ani Nenkova EMNLP 2025 (Findings)
• DocVoyager: Anticipating Users' Information Needs and Guiding Document Reading through Question Answering Yoonjoo Lee, Nedim Lipka, Zichao Wang, Ryan Rossi, Puneet Mathur, Tong Sun, Alexa Siu CHI 2025
• VisDoM: Multi-Document QA with Visually Rich Elements Using Multimodal Retrieval-Augmented Generation Manan Suri, Puneet Mathur, Franck Dernoncourt, Kanika Goswami, Ryan A. Rossi, Dinesh Manocha NAACL 2025
• MoDS: Moderating a Mixture of Document Speakers to Summarize Debatable Queries in Document Collections Nishant Balepur, Alexa Siu, Nedim Lipka, Franck Dernoncourt, Tong Sun, Jordan Lee Boyd-Graber, Puneet Mathur NAACL 2025
• MATSA: Multi-Agent Table Structure Attribution Puneet Mathur, Alexa Siu, Nedim Lipka, Tong Sun EMNLP 2024
• DocEdit-v2: Document Structure Editing Via Multimodal LLM Grounding Manan Suri, Puneet Mathur, Franck Dernoncourt, Rajiv Jain, Vlad I Morariu, Ramit Sawhney, Preslav Nakov, Dinesh Manocha EMNLP 2024
• Agent-DocEdit: Language-Instructed LLM Agent for Content-Rich Document Editing Te-Lin Wu, Rajiv Jain, Yufan Zhou, Puneet Mathur, Vlad I Morariu COLM 2024
• DocPilot: Copilot for Automating PDF Edit Workflows in Documents Puneet Mathur, Alexa Siu, Varun Manjunatha, Tong Sun ACL 2024
• DocScript: Document-level Script Event Prediction Puneet Mathur, Rajiv Jain, Vlad Morariu, Aparna Garimella, Franck Dernoncourt, Jiuxiang Gu, Ramit Sawhney, Preslav Nakov, Dinesh Manocha COLING 2024
• DocEdit: Language-guided Document Editing Puneet Mathur, Rajiv Jain, Jiuxiang Gu, Franck Dernoncourt, Dinesh Manocha, Vlad Morariu AAAI 2023
• LayerDoc: Layer-wise Extraction of Spatial Hierarchical Structure in Visually-Rich Documents Puneet Mathur, Rajiv Jain, Ashutosh Mehra, Jiuxiang Gu, Franck Dernoncourt, Anandhavelu N, Quan Tran, Verena Kaynig-Fittkau, Ani Nenkova, Dinesh Manocha, Vlad Morariu WACV 2023
• DocInfer: Document-level Natural Language Inference using Optimal Evidence Selection Puneet Mathur, Gautam Kunapuli, Riyaz Ahmad Bhat, Manish Shrivastava, Dinesh Manocha, Maneesh Singh EMNLP 2022
• DocFin: Multimodal Financial Prediction and Bias Mitigation using Semi-structured Documents Puneet Mathur, Mihir Goyal, Ramit Sawhney, Ritik Mathur, Jochen L. Leidner, Franck Dernoncourt, Dinesh Manocha EMNLP 2022 (Findings)
• DocTime: A Document-level Temporal Dependency Graph Parser Puneet Mathur, Vlad I Morariu, Verena Kaynig-Fittkau, Jiuxiang Gu, Franck Dernoncourt, Quan Hung Tran, Ani Nenkova, Dinesh Manocha, Rajiv Jain NAACL 2022
• TIMERS: Document-level Temporal Relation Extraction Puneet Mathur, Rajiv Jain, Franck Dernoncourt, Vlad Morariu, Quan Hung Tran, Dinesh Manocha ACL 2021

Multimodal AI, Affective Computing & Video Understanding

• Saliency-Aware Interpolative Augmentation for Multimodal Financial Prediction Samyak Jain, Parth Chhabra, Atula Neerkaje, Puneet Mathur, Ramit Sawhney, Shivam Agarwal, Preslav Nakov, Sudheer Chava, Dinesh Manocha COLING 2024
• MONOPOLY: Financial Prediction from MONetary POLicY Conference Videos Using Multimodal Cues Puneet Mathur, Atula Tejaswi Neerkaje, Malika Chhibber, Ramit Sawhney, Fu-Ming Guo, Franck Dernoncourt, Sanghamitra Dutta, Dinesh Manocha ACM Multimedia 2022
• 3MASSIV: Multilingual, Multimodal and Multi-Aspect Dataset of Social Media Short Videos Vikram Gupta, Trisha Mittal, Puneet Mathur, Vaibhav Mishra, Mayank Maheshwari, Aniket Bera, Debdoot Mukherjee, Dinesh Manocha Dataset CVPR 2022
• Multimodal Multi-Speaker Merger and Acquisition (M3A) Financial Forecasting: A New Task, Dataset, and Neural Baselines Ramit Sawhney, Mihir Goyal, Prakhar Goel, Puneet Mathur, Rajiv Ratn Shah ACL 2021
• Affect2MM: Affective Analysis of Multimedia Content Using Emotion Causality Trisha Mittal, Puneet Mathur, Aniket Bera, Dinesh Manocha CVPR 2021
• Multitask Learning for Emotionally Analyzing Sexual Abuse Disclosures Ramit Sawhney, Puneet Mathur, Taru Jain, Akash Kumar Gautam, Rajiv Ratn Shah NAACL 2021
• Dynamic Graph Modeling of Simultaneous EEG and Eye-tracking Data For Reading Task Identification Puneet Mathur, Trisha Mittal, Dinesh Manocha ICASSP 2021
• Multimodal Multitask Financial Risk Forecasting Ramit Sawhney, Puneet Mathur, Piyush Khanna, Ayush Mangal, Rajiv Ratn Shah ACM Multimedia 2020 (Oral)
• PersonaLM: Language Model Personalization via Domain-distributed Span Aggregated K-Nearest N-gram Retrieval Augmentation Puneet Mathur, Zhe Liu, Ke Li, Yingyi Ma, Gil Keren, Zeeshan Ahmed, Dinesh Manocha, Xuedong Zhang EMNLP 2023 (Findings)
• DOC-RAG: ASR Language Model Personalization with Domain-Distributed Co-occurrence Retrieval Augmentation Puneet Mathur, Zhe Liu, Ke Li, Yingyi Ma, Gil Keren, Zeeshan Ahmed, Dinesh Manocha, Xuedong Zhang COLING 2024
• DocLayoutTTS: Dataset and Baselines for Layout-informed Document-level Neural Speech Synthesis Puneet Mathur, Franck Dernoncourt, Quan Hung Tran, Jiuxiang Gu, Ani Nenkova, Vlad Morariu, Rajiv Jain, Dinesh Manocha Interspeech 2022
• Risk Forecasting from Earnings Calls Acoustics and Network Correlations Ramit Sawhney, Arshiya Aggarwal, Piyush Khanna, Puneet Mathur, Taru Jain, Rajiv Ratn Shah Interspeech 2020
• VolTAGE: Volatility Forecasting via Text Audio Fusion with Graph Convolution Networks for Earnings Calls Ramit Sawhney, Arshiya Aggarwal, Piyush Khanna, Taru Jain, Puneet Mathur, Rajiv Ratn Shah EMNLP 2020
• Meta-learning for Low-Resource Speech Emotion Recognition Suransh Chopra*, Puneet Mathur*, Ramit Sawhney, Rajiv Ratn Shah ICASSP 2021