Qitian Wu (吴齐天)

Bio

Qitian is currently a postdoctoral fellow of Eric and Wendy Schmidt Center at Broad Institute of MIT and Harvard. Prior to this, he finished the PhD in Computer Science from Shanghai Jiao Tong University (SJTU), Before that, he achieved the Bachelor (Microelectronics, Mathematics) and Master (Computer Science) degrees from SJTU, and worked as research intern at Tencent WeChat, Amazon Web Service and BioMap.

His research interest predominantly revolves around machine learning foundations and applications. On the foundation side, he builds theoretically principled and practically useful methodology, particularly for learning with complex structures and distribution shifts. He also explores the intersection with interdisciplinary areas such as biomedical sciences and recommender systems, and sought inspirations from physics. He is the recipients of Microsoft PhD Fellowship, Baidu PhD Fellowship and Scholar Star at SJTU.

Research Summary

My research generally aims at improving and broadening the capabilities of AI models to complex structured data. My recent works unfold under two learning settings, i.e., closed-world and open-world hypothesis, along which my works explore the methodology for representation and generalization challenges.

In the closed-world regime, wherein training and test data distributions are overlapped, my research targets building foundation models as general-purpose encoder backbones (e.g., Transformers) for obtaining high-quality representation. My recent works particularly focus on the scalability and interpretability, two key challenges concerning building foundation models for data involving geometries and structures.
- For scalability, our works (NodeFormer on NeurIPS'22 and SGFormer on NeurIPS'23) explore designing scalable and efficient models that can leverage all-pair global interactions with the cost of linear complexity.
- For interpretability, our work (DIFFormer on ICLR'23) probes into the analogy between neural message passing and energy-constrained diffusion systems, yielding a physics-inspired interpretable framework for architectural designs.
In the open-world regime, wherein training and test data come from disparate distributions, the research focus is how to handle the distribution shifts in the wild. My recent works endeavor to understand the generalization limits of neural networks and develop provably generalizable learning algorithms to address various distribution shifts.
- For studying the generalizability, our work (EERM on ICLR'22) formulates the out-of-distribution generalization problem on graphs and proposes a provably generalizable learning algorithm with invariance principles. From another standpoint, our work (CaNet on WWW'24) discusses how causal intervention can be utilized for guiding generalization, and the follow-up work (GLIND on ICML'24) further extends the causal perspective to geometric diffusion systems.
- For enhancing the robustness, our work (GNNSafe on ICLR'23) targets how to endow the pre-trained model with capabilities for identifying data from distributions different than training, i.e., out-of-distribution detection.

I am always open for collaborations, and if you are interested, feel free to have a chat.

Publications

The most recent works can be found on Google Scholar.

Selected
Representative
All (in chronological order)

DIFFormer: Scalable (Graph) Transformers Induced by Energy Constrained Diffusion

Qitian Wu, Chenxiao Yang, Wentao Zhao, Yixuan He, David Wipf and Junchi Yan

International Conference on Learning Representations (ICLR) 2023 oral presentation, ranking among top 0.5%

Paper Slides Poster Video Code Blog (Chinese)

Summary: We propose a geometric diffusion framework with energy constraints and show its solution aligns with widely used attention networks, upon which we propose diffusion-based Transformers.

NodeFormer: A Scalable Graph Structure Learning Transformer for Node Classification

Qitian Wu, Wentao Zhao, Zenan Li, David Wipf and Junchi Yan

Advances in Neural Information Processing Systems (NeurIPS) 2022 spotlight presentation

Paper Slides Poster Video Code Blog (Chinese) Blog (English)

Summary: We propose a scalable graph Transformer with efficient all-pair message passing achieved in O(N) complexity. The global attention over 2M nodes only requires 4GB memory.

Handling Distribution Shifts on Graphs: An Invariance Perspective

Qitian Wu, Hengrui Zhang, Junchi Yan and David Wipf

International Conference on Learning Representations (ICLR) 2022

Paper Slides Poster Code Blog (Chinese) Blog (English)

Summary: We formulate out-of-distribution generalization on graphs and discuss how to leverage (causal) invariance principle for handling graph-based distribution shifts.

Learning Divergence Fields for Shift-Robust Message Passing

Qitian Wu, Fan Nie, Chenxiao Yang and Junchi Yan

International Conference on Machine Learning (ICML) 2024

Paper Slides Code Blog (Chinese) Blog (English)

Graph Out-of-Distribution Generalization via Causal Intervention

Qitian Wu, Fan Nie, Chenxiao Yang, Tianyi Bao and Junchi Yan

The Web Conference (WWW) 2024 oral presentation

Paper Slides Code Blog (Chinese) Blog (English)

SGFormer: Simplifying and Empowering Transformers for Large-Graph Representations

Qitian Wu, Wentao Zhao, Chenxiao Yang, Hengrui Zhang, Fan Nie, Haitian Jiang, Yatao Bian and Junchi Yan

Advances in Neural Information Processing Systems (NeurIPS) 2023

Paper Slides Code Blog (Chinese)

Summary: We propose an efficient Transformer that only uses one-layer global attention and significantly reduces computation cost for large-graph representations.

DIFFormer: Scalable (Graph) Transformers Induced by Energy Constrained Diffusion

Qitian Wu, Chenxiao Yang, Wentao Zhao, Yixuan He, David Wipf and Junchi Yan

International Conference on Learning Representations (ICLR) 2023 oral presentation, ranking among top 0.5%

Paper Slides Poster Video Code Blog (Chinese)

Summary: We propose a geometric diffusion framework with energy constraints and show its solution aligns with widely used attention networks, upon which we propose diffusion-based Transformers.

Energy-based Out-of-Distribution Detection for Graph Neural Networks

Qitian Wu, Yiting Chen, Chenxiao Yang, and Junchi Yan

International Conference on Learning Representations (ICLR) 2023

Paper Slides Poster Code Blog (Chinese)

Summary: We extract a OOD detection model from GNN classifier through energy-based models and energy-based belief propagation, reducing FPR95 over SOTAs by up to 44.8%.

Graph Neural Networks are Inherently Good Generalizers: Insights by Bridging GNNs and Multi-Layer Perceptrons

Chenxiao Yang, Qitian Wu, Jiahua Wang and Junchi Yan

International Conference on Learning Representations (ICLR) 2023

Paper Code Blog (Chinese)

Summary: We identify that the efficacy of GNNs over MLP mostly stems from its inherent better generalization, demonstrated by experiments on sixteen benchmarks and neural tangent kernel theory.

NodeFormer: A Scalable Graph Structure Learning Transformer for Node Classification

Qitian Wu, Wentao Zhao, Zenan Li, David Wipf and Junchi Yan

Advances in Neural Information Processing Systems (NeurIPS) 2022 spotlight presentation

Paper Slides Poster Video Code Blog (Chinese) Blog (English)

Summary: We propose a scalable graph Transformer with efficient all-pair message passing achieved in O(N) complexity. The global attention over 2M nodes only requires 4GB memory.

Learning Substructure Invariance for Out-of-Distribution Molecular Representations

Nianzu Yang, Kaipeng Zeng, Qitian Wu, Xiaosong Jia and Junchi Yan

Advances in Neural Information Processing Systems (NeurIPS) 2022 spotlight presentation

Paper Code Blog (Chinese)

Summary: We propose a invariant learning approach for molecular property prediction under distribution shifts and achieve SOTA results on OGB-mol and DrugOOD benchmarks.

Geometric Knowledge Distillation: Topology Compression for Graph Neural Networks

Chenxiao Yang, Qitian Wu and Junchi Yan

Advances in Neural Information Processing Systems (NeurIPS) 2022

Paper Code Blog (Chinese)

Summary: We explore geometric knowledge distillation, empowered by neural heat kernel, that can generalize the topological knowledge from larger GNNs to smaller one.

Towards Out-of-Distribution Sequential Event Prediction: A Causal Treatment

Chenxiao Yang, Qitian Wu, Qingsong Wen, Zhiqiang Zhou, Liang Sun and Junchi Yan

Advances in Neural Information Processing Systems (NeurIPS) 2022

Paper Code Blog (Chinese)

Summary: We use causal analysis to reveal the limitations of Maximum Likelihood Estimation for sequential prediction under distribution shifts and propose an effective treatment based on backdoor adjustment.

Handling Distribution Shifts on Graphs: An Invariance Perspective

Qitian Wu, Hengrui Zhang, Junchi Yan and David Wipf

International Conference on Learning Representations (ICLR) 2022

Paper Slides Poster Code Blog (Chinese) Blog (English)

Summary: We formulate out-of-distribution generalization on graphs and discuss how to leverage (causal) invariance principle for handling graph-based distribution shifts.

Towards Open-World Recommendation: An Inductive Model-based Collaborative Filtering Approach

Qitian Wu, Hengrui Zhang, Xiaofeng Gao, Junchi Yan and Hongyuan Zha

International Conference on Machine Learning (ICML) 2021 spotlight presentation

Paper Slides Poster Code Blog (Chinese)

Summary: We propose a latent structure inference model to handle new unseen users in the testing phase of recommender systems.

Towards Open-World Feature Extrapolation: An Inductive Graph Learning Approach

Qitian Wu, Chenxiao Yang and Junchi Yan

Advances in Neural Information Processing Systems (NeurIPS) 2021

Paper Slides Poster Code Blog (Chinese)

Summary: We propose a graph representation learning approach for handling feature space expansion from training data to testing data.

From Canonical Correlation Analysis to Self-supervised Graph Neural Networks

Hengrui Zhang, Qitian Wu, Junchi Yan, David Wipf and Philip S. Yu

Advances in Neural Information Processing Systems (NeurIPS) 2021

Paper Code

Summary: We introduce a simple-yet-effective contrastive objective for self-supervised learning on graphs.

Bridging Explicit and Implicit Deep Generative Models via Neural Stein Estimators

Qitian Wu, Han Gao and Hongyuan Zha

Advances in Neural Information Processing Systems (NeurIPS) 2021

Paper Code

Summary: We leverage the Stein's distance as a novel tool to construct a theoretically grounded bridging term between two families of deep generative models.

Dual Graph Attention Networks for Deep Latent Representation of Multifaceted Social Effects in Recommender Systems

Qitian Wu, Hengrui Zhang, Xiaofeng Gao, Peng He, Paul Weng, Han Gao and Guihai Chen

The Web Conference (WWW) 2019 long oral representation

Paper Code

Summary: We propose a dual graph attention model that captures the social homophily and influence effects among users and items in recommender systems.

Learning Divergence Fields for Shift-Robust Message Passing

Qitian Wu, Fan Nie, Chenxiao Yang and Junchi Yan

International Conference on Machine Learning (ICML) 2024

How Graph Neural Networks Learn: Lessons from Training Dynamics

Chenxiao Yang, Qitian Wu, David Wipf, Ruoyu Sun and Junchi Yan

International Conference on Machine Learning (ICML) 2024

Graph Out-of-Distribution Detection Goes Neighborhood Shaping

Tianyi Bao, Qitian Wu, Zetian Jiang, Yiting Chen, Jiawei Sun and Junchi Yan

International Conference on Machine Learning (ICML) 2024

Graph Out-of-Distribution Generalization via Causal Intervention

Qitian Wu, Fan Nie, Chenxiao Yang, Tianyi Bao and Junchi Yan

The Web Conference (WWW) 2024 oral presentation

Rethinking Cross-Domain Sequential Recommendation Under Open-World Assumptions

Wujiang Xu, Qitian Wu, Runzhong Wang, Mingming Ha, Qiongxu Ma, Linxun Chen, Bing Han and Junchi Yan

The Web Conference (WWW) 2024

SGFormer: Simplifying and Empowering Transformers for Large-Graph Representations

Qitian Wu, Wentao Zhao, Chenxiao Yang, Hengrui Zhang, Fan Nie, Haitian Jiang, Yatao Bian and Junchi Yan

Advances in Neural Information Processing Systems (NeurIPS) 2023

Unleashing the Power of Graph Data Augmentation on Covariate Distribution Shift

Yongduo Sui, Qitian Wu, Jiancan Wu, Qing Cui, Longfei Li, Jun Zhou, Xiang Wang, Xiangnan He

Advances in Neural Information Processing Systems (NeurIPS) 2023

GraphGlow: Universal and Genralizable Structure Learning for Graph Neural Networks

Wentao Zhao, Qitian Wu, Chenxiao Yang and Junchi Yan

ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) 2023

MoleRec: Combinatorial Drug Recommendation with SubstructureAware Molecular Representation Learning

Nianzu Yang, Kaipeng Zeng, Qitian Wu, Junchi Yan

The Web Conference (WWW) 2023