avatar2.jpg

Liangming Pan (潘亮铭)

Assistant Professor
College of Information Science
University of Arizona

Hey, thanks for stopping by! 👋

I am an Assistant Professor at the College of Information Science, University of Arizona. Previously, I was a postdoctoral scholar at the Natural Language Processing Group, University of California, Santa Barbara (UCSB), where I was fortunate to be advised by Prof. William Yang Wang. I completed my PhD from National University of Singapore in 2022, where I was advised by Prof. Min-Yen Kan and Prof. Tat-Seng Chua and worked on complex question answering and generation. Previously, I received my Master’s degree from Tsinghua University (2014 - 2017, advised by Prof. Juanzi Li and Prof. Jie Tang) and my Bachelor’s degree from Beihang University (2010 - 2014).

My primary research lies in the area of natural language processing and machine learning. The overarching goal of my research is: how to build trustworthy large language models that are logical, truthful, and safe. Specifically:

  • Reasoning: How/why can LLMs consistently perform logical and faithful reasoning?
  • Truthfulness: How can LLMs produce accurate, un-biased, and up-to-date information?
  • Safety: How can we discover and mitigate the potential harm caused by LLMs?
📢 Annoncement

I am looking for highly motivated Ph.D./master/undergraduate students to work with me on exciting research projects related to large language models. If you are interested in working with me, please feel free to reach out to me via email.


News

  Oct, 2024   5 papers accepted by EMNLP 2024 (2 Main Conference, 3 Findings). Topics involves logical reasoning, automated fact-checking, multi-agent reasoning, and knowledge editing. Our paper MMLongBench-Doc was accepted as spotlight paper in NeurIPS 2024 (Dataset and Benchmark).
  Jul, 2024   I will serve as an Area Chair for EMNLP 2024 and COLING 2025.
  Jun, 2024   Invited talk “Empowering Large Language Models with Faithful Reasoning” at Tsinghua University, Peking University, Xi’an Jiaotong University, and Harbin Institute of Technology (Shenzhen).
  May, 2024   6 papers accepted by ACL 2024 (2 Main Conference, 4 Findings). Topics involves logical reasoning, uncertainty estimation, and self-correction of LLMs.

2 papers accepted by ICML 2024. One paper is on understanding the chain-of-thought reasoning ability of LLMs. The other paper is on analyzing the impact of social media influencers on AI research visibility.
  Apr, 2024   I will serve as the Student Volunteer Chair and an Area Chair of ACL 2024.
  Dec, 2023   Invited talk “Combating Misinformation in the age of LLMs” at NUS Centre for Trusted Internet and Community. [Slides]
  Nov, 2023   Our paper Attacking Open-domain Question Answering by Injecting Misinformation received the Area Chair Award (Question Answering) at IJCNLP-AACL 2023.
  Oct, 2023   I have 8 papers accepted by EMNLP 2023 (4 Main Conference, 3 Findings, 1 Demo). Topics involves logical reasoning, safety, and evaluation of LLMs.
  Aug, 2023   New Survey Paper! Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies. We also create a paper list.
  May, 2023   New Preprint! Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning.

Our paper Fact-Checking Complex Claims with Program-Guided Reasoning was accepted by the main conference of ACL 2023.
  Feb, 2023   Our paper Hashtag-Guided Low-Resource Tweet Classification was accepted by WWW 2023.

Invited Talk “Building Data-efficient and Explainable Fact-Checking Models” at Mohamed bin Zayed University of Artificial Intelligence (MBZUAI).
  Dec, 2022   I will serve as an Area Chair for the Question Answering track of ACL 2023.

Starting from Dec 2022, I am thrilled to join the UC Santa Barbara Natural Language Processing Group as a Postdoctoral Scholar, working with Prof. William Yang Wang.


Selected Publications

  1. MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizations
    Yubo Ma, Yuhang Zang, Liangyu Chen, Meiqi Chen, Yizhu Jiao, Xinze Li, Xinyuan Lu, Ziyu Liu, Yan Ma, Xiaoyi Dong, Pan Zhang,  Liangming Pan, Yu-Gang Jiang, Jiaqi Wang, Yixin Cao, and Aixin Sun
    In Annual Conference on Neural Information Processing Systems (NeurIPS) (Dataset and Benchmark Track), 2024
    Spotlight Paper
  2. AKEW: Assessing Knowledge Editing in the Wild
    Xiaobao Wu,  Liangming Pan, William Yang Wang, and Anh Tuan Luu
    In Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024
  3. A Survey on Detection of LLMs-Generated Content
    Xianjun Yang,  Liangming Pan, Xuandong Zhao, Haifeng Chen, Linda Petzold, William Yang Wang, and Wei Cheng
    In Findings of Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024
  4. Automatically Correcting Large Language Models: Surveying the Landscape of Diverse Automated Correction Strategies
    Liangming Pan, Michael Saxon, Wenda Xu, Deepak Nathani, Xinyi Wang, and William Yang Wang
    Transactions of the Association for Computational Linguistics (TACL), 2024
    Oral Presentation at ACL 2024
  5. ACL
    Faithful Logical Reasoning via Symbolic Chain-of-Thought
    Jundong Xu, Hao Fei,  Liangming Pan, Qian Liu, Mong-Li Lee, and Wynne Hsu
    In Annual Meeting of the Association for Computational Linguistics (ACL), 2024
  6. ACL Oral Presentation
    Pride and Prejudice: LLM Amplifies Self-Bias in Self-Refinement
    Wenda Xu, Guanglei Zhu, Xuandong Zhao,  Liangming Pan, Lei Li, and William Yang Wang
    In Annual Meeting of the Association for Computational Linguistics (ACL), 2024
  7. ACL
    Knowledge of Knowledge: Exploring Known-Unknowns Uncertainty with Large Language Models
    Alfonso Amayuelas, Kyle Wong,  Liangming Pan, Wenhu Chen, and William Yang Wang
    In Findings of Annual Meeting of the Association for Computational Linguistics (ACL), 2024
  8. ACL
    The Knowledge Alignment Problem: Bridging Human and External Knowledge for Large Language Models
    Shuo Zhang,  Liangming Pan, Junzhou Zhao, and William Yang Wang
    In Findings of Annual Meeting of the Association for Computational Linguistics (ACL), 2024
  9. Understanding Reasoning Ability of Language Models From the Perspective of Reasoning Paths Aggregation
    Xinyi Wang, Alfonso Amayuelas, Kexun Zhang,  Liangming Pan, Wenhu Chen, and William Yang Wang
    In International Conference on Machine Learning (ICML), 2024
  10. SCITAB: A Challenging Benchmark for Compositional Reasoning and Claim Verification on Scientific Tables
    Xinyuan Lu*Liangming Pan*, Qian Liu, Preslav Nakov, and Min-Yen Kan
    In Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
  11. MAF: Multi-Aspect Feedback for Improving Reasoning in Large Language Models
    Deepak Nathani, David Wang,  Liangming Pan, and William Wang
    In Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
  12. EMNLP Oral Presentation
    INSTRUCTSCORE: Towards Explainable Text Generation Evaluation with Automatic Feedback
    Wenda Xu, Danqing Wang,  Liangming Pan, Zhenqiao Song, Markus Freitag, William Wang, and Lei Li
    In Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
  13. Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning
    Liangming Pan, Alon Albalak, Xinyi Wang, and William Wang
    In Findings of Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
  14. On the Risk of Misinformation Pollution with Large Language Models
    Yikang Pan*Liangming Pan*, Wenhu Chen, Preslav Nakov, Min-Yen Kan, and William Wang
    In Findings of Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
  15. QACheck: A Demonstration System for Question-Guided Multi-Hop Fact-Checking
    Liangming Pan, Xinyuan Lu, Min-Yen Kan, and Preslav Nakov
    In Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP Demo), 2023
  16. ACL
    Fact-Checking Complex Claims with Program-Guided Reasoning
    Liangming Pan, Xiaobao Wu, Xinyuan Lu, Anh Tuan Luu, William Yang Wang, Min-Yen Kan, and Preslav Nakov
    In Annual Meeting of the Association for Computational Linguistics (ACL), 2023
  17. AACL / IJCNLP Oral Presentation
    Attacking Open-domain Question Answering by Injecting Misinformation
    Liangming Pan, Wenhu Chen, Min-Yen Kan, and William Yang Wang
    In International Joint Conference on Natural Language Processing and Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL), 2023
    Area Chair Award (Question Answering Track)
  18. AACL / IJCNLP Oral Presentation
    Investigating Zero- and Few-shot Generalization in Fact Verification
    Liangming Pan, Yunxiang Zhang, and Min-Yen Kan
    In International Joint Conference on Natural Language Processing and Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL), 2023
  19. AACL / IJCNLP Oral Presentation
    FollowupQG: Towards Information-Seeking Follow-up Question Generation
    Yan Meng,  Liangming Pan, Yixin Cao, and Min-Yen Kan
    In International Joint Conference on Natural Language Processing and Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL), 2023
  20. Efficient Online Data Mixing For Language Model Pre-Training
    Alon Albalak,  Liangming Pan, Colin Raffel, and William Yang Wang
    In NeurIPS Workshop on Robustness of Few-shot and Zero-shot Learning in Foundation Models (R0-FoMo@NeurIPS), 2023
    Spotlight Paper
  21. ACL Oral Presentation
    KQA Pro: A Dataset with Explicit Compositional Programs for Complex Question Answering over Knowledge Base
    Shulin Cao, Jiaxin Shi,  Liangming Pan, Lunyiu Nie, Yutong Xiang, Lei Hou, Juanzi Li, Bin He, and Hanwang Zhang
    In Annual Meeting of the Association for Computational Linguistics (ACL), 2022
  22. ACL
    Interpreting the Robustness of Neural NLP Models to Textual Perturbations
    Yunxiang Zhang,  Liangming Pan, Samson Tan, and Min-Yen Kan
    In Findings of Annual Meeting of the Association for Computational Linguistics (ACL), 2022
  23. ACL Oral Presentation
    Zero-shot Fact Verification by Claim Generation
    Liangming Pan, Wenhu Chen, Wenhan Xiong, Min-Yen Kan, and William Yang Wang
    In Annual Meeting of the Association for Computational Linguistics (ACL), 2021
  24. NAACL Oral Presentation
    Unsupervised Multi-hop Question Answering by Question Generation
    Liangming Pan, Wenhu Chen, Wenhan Xiong, Min-Yen Kan, and William Yang Wang
    In Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2021
  25. ACL Oral Presentation
    Semantic Graphs for Generating Deep Questions
    Liangming Pan, Yuxi Xie, Yansong Feng, Tat-Seng Chua, and Min-Yen Kan
    In Annual Meeting of the Association for Computational Linguistics (ACL), 2020
  26. ACL Oral Presentation
    Expertise Style Transfer: A New Task Towards Better Communication between Experts and Laymen
    Yixin Cao, Ruihao Shui,  Liangming Pan, Min-Yen Kan, Zhiyuan Liu, and Tat-Seng Chua
    In Annual Meeting of the Association for Computational Linguistics (ACL), 2020
  27. Exploring and Evaluating Attributes, Values, and Structures for Entity Alignment
    Zhiyuan Liu, Yixin Cao,  Liangming Pan, Juanzi Li, Zhiyuan Liu, and Tat-Seng Chua
    In Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020
  28. Exploring Question-Specific Rewards for Generating Deep Questions
    Yuxi Xie,  Liangming Pan*, Dongzhe Wang, Min-Yen Kan, and Yansong Feng
    In International Conference on Computational Linguistics (COLING), 2020
  29. ACM MM Oral Presentation
    Multi-modal Cooking Workflow Construction for Food Recipes
    Liangming Pan, Jingjing Chen, Jianlong Wu, Shaoteng Liu, Chong-Wah Ngo, Min-Yen Kan, Yu-Gang Jiang, and Tat-Seng Chua
    In ACM International Conference on Multimedia (ACM MM), 2020
  30. TMM
    A Hybrid Approach for Detecting Prerequisite Relations in Multi-Modal Food Recipes
    Liangming Pan, Jingjing Chen, Shaoteng Liu, Chong-Wah Ngo, Min-Yen Kan, and Tat-Seng Chua
    IEEE Transactions on Multimedia (TMM), 2021
  31. CVPR
    Hyperbolic Visual Embedding Learning for Zero-Shot Recognition
    Shaoteng Liu, Jingjing Chen,  Liangming Pan, Chong-Wah Ngo, Tat-Seng Chua, and Yu-Gang Jiang
    In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020
  32. Zero-Shot Ingredient Recognition by Multi-Relational Graph Convolutional Network
    Jingjing Chen,  Liangming Pan, Zhipeng Wei, Xiang Wang, Chong-Wah Ngo, and Tat-Seng Chua
    In AAAI Conference on Artificial Intelligence (AAAI), 2020
  33. ACL
    Prerequisite Relation Learning for Concepts in MOOCs
    Liangming Pan, Chengjiang Li, Juanzi Li, and Jie Tang
    In Annual Meeting of the Association for Computational Linguistics (ACL), 2017
  34. Course Concept Extraction in MOOCs via Embedding-Based Graph Propagation
    Liangming Pan, Xiaochen Wang, Chengjiang Li, Juanzi Li, and Jie Tang
    In International Joint Conference on Natural Language Processing (IJCNLP), 2017