Research | Liangming Pan

My research focuses on natural language processing, machine learning, large language model, and explainable AI. Our goal is to uncover the mechanisms behind large language model and use that understanding to build trustworthy models that are reliable, truthful, and safe.

Mechanistic Study of LLMs

How can we open the black box of LLMs to uncover their internal mechanisms, especially those enabling complex reasoning?

Mechanism-Guided Trustworthy LLM

How can insights from mechanistic understanding be translated into practice for building LLMs that are reliable, truthful, and safe in real-world applications?

Featured Research

How does Transformer Learn Implicit Reasoning?

Jiaran Ye, Zijun Yao, Zhidian Huang, Liangming Pan, Jinxin Liu, Yushi Bai, Amy Xin, Liu Weichuan, Xiaoyin Che, Lei Hou, Juanzi Li

arXiv 2025

Paper

How Is LLM Reasoning Distracted by Irrelevant Context? An Analysis Using a Controlled Benchmark

Minglai Yang, Ethan Huang, Liang Zhang, Mihai Surdeanu, William Wang, Liangming Pan

ACL 2025

Mechanistic Study of LLMs

Mechanism-Guided Trustworthy LLM

Featured Research

How does Transformer Learn Implicit Reasoning?

How Is LLM Reasoning Distracted by Irrelevant Context? An Analysis Using a Controlled Benchmark

AntiLeak-Bench: Preventing Data Contamination by Automatically Constructing Benchmarks with Updated Real-World Knowledge

Aristotle: Mastering Logical Reasoning with A Logic-Complete Decompose-Search-Resolve Framework

Gödel Agent: A Self-Referential Agent Framework for Recursive Self-Improvement

CausalEval: Towards Better Causal Reasoning in Language Models

TART: An Open-Source Tool-Augmented Framework for Explainable Table-based Reasoning

MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizations

Automatically Correcting Large Language Models: Surveying the Landscape of Diverse Automated Correction Strategies

Faithful Logical Reasoning via Symbolic Chain-of-Thought

Understanding Reasoning Ability of Language Models From the Perspective of Reasoning Paths Aggregation

SCITAB: A Challenging Benchmark for Compositional Reasoning and Claim Verification on Scientific Tables

Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning

On the Risk of Misinformation Pollution with Large Language Models

Fact-Checking Complex Claims with Program-Guided Reasoning

Attacking Open-domain Question Answering by Injecting Misinformation