Qitian Wu (吴齐天)

What's News

[2024.05] Three papers were accepted to ICML, congrats to Chenxiao and Tianyi.

[2024.01] Two papers (with one selected as Oral) were accepted to WWW. See you in Singapore.

[2023.12] Honored to be awarded with the Academic Scholar Star in SJTU.

[2023.10] Gave a talk on physics-inspired learning with non-IID data at ByteDance AI Lab.

[2023.09] Two papers were accepted to NeurIPS2023. See you in New Orleans.

[2023.09] Honored to be awarded with National PhD Scholarship.

[2023.07] Gave a talk on graph Transformers on LOG seminar (in Chinese). See the video here.

[2023.05] One paper is accepted to KDD2023, congrats to Wentao.

[2023.03] Gave a talk on learning on graphs with open world assumptions at Bosch AI Center.

[2023.02] One paper about combinatorial drug recommendation is accepted to WWW2023, congrats to Nianzu.

[2023.02] Gave a talk on graph Transformers at AI Times.

[2023.01] Three papers with one spotlight (notably-top-25%) were accepted to ICLR 2023, congrats to Chenxiao.

[2022.11] Two papers are selected as spotlight presentation (less than 5%) on NeurIPS 2022!

[2022.10] I was awarded with National PhD Scholarship (top 1%).

[2022.10] I will give a talk on graph OOD generalization at LoG seminar.

[2022.09] Five papers on OOD generalization and graph Transformers were accepted to NeurIPS'22.

[2022.05] Two papers on GNNs and causal learning were accepted to SIGKDD'22.

[2022.04] One paper on negative sampling was accepted to IJCAI'22.

[2022.03] I will give a talk on out-of-distribution generalization at AI Drive PaperWeekly.

[2022.01] One paper on learning with graph distribution shifts was accepted to ICLR'22.

[2021.12] I will give a talk on open-world learning at Baiyulan young researcher forum.

[2021.11] I was awarded with Baidu Scholarship (only 10 from worldwide).

[2021.10] I was awarded with Microsoft Research PhD Fellowship (only 11 from Asia).

[2021.09] Three papers were accepted to NeurIPS'21.

[2021.08] One paper on sequential recommendation was accepted to CIKM'21 as spotlight.

[2021.04] I was elected as Global Top 100 AI Rising Star!

Bio

I am a final-year PhD student at the Department of Computer Science and Engineering from Shanghai Jiao Tong University (SJTU), advised by Junchi Yan. I also extensively collaborated with David Wipf and Hongyuan Zha. I achieved the Bachelor (Microelectronics, Mathematics) and Master (Computer Science) degrees from SJTU, and worked as research intern at Tencent WeChat, Amazon AI Lab and BioMap.

My research interest predominantly revolves around machine learning foundations and applications. On the foundation side, I build theoretically principled and practically useful methodology, particularly for learning with complex structures and distribution shifts. I also explore the intersection with interdisciplinary areas such as life sciences (e.g., drug discovery and healthcare) and recommender systems, and sought inspirations from physics. My research is supported by Microsoft PhD Fellowship and Baidu PhD Fellowship.

Research Summary

My recent works aim at machine learning with complex structured data, especially making the models more expressive, generalizable and reliable. in both closed-world and open-world regimes.

In the closed-world regime, wherein training and test data distributions are overlapped, my research focus is how to build foundation models as general-purpose encoder backbones (e.g., Transformers) for obtaining high-quality representation. My recent works particularly focus on the scalability and interpretability, two key challenges concerning building foundation models for data involving geometries and structures.
- For scalability, our works (NodeFormer on NeurIPS'22 and SGFormer on NeurIPS'23) explore designing scalable and efficient models that can discover and leverage all-pair global interactions with the cost of linear complexity.
- For interpretability, our work (DIFFormer on ICLR'23) probes into the analogy between neural message passing and energy-constrained diffusion systems, yielding a physics-inspired interpretable framework for architectural designs.
In the open-world regime, wherein training and test data come from disparate distributions, the research focus is how to handle the distribution shifts in the wild. My recent works endeavor to understand the generalization limits of neural networks and develop provably generalizable learning algorithms to address distribution shifts.
- For studying the generalizability, our work (EERM on ICLR'22) formulates the out-of-distribution generalization problem on graphs and proposes a provably generalizable learning algorithm with causal invariance principles. The proposed methodology is applied to handling distribution shifts in molecular discovery (MoleOOD on NeurIPS'22) and recommender systems (Caseq on NeurIPS'22).
- For enhancing the robustness, our work (GNNSafe on ICLR'23) targets how to endow the pre-trained model with capabilities for identifying data from distributions different than training, i.e., out-of-distribution detection.

Publications

The most recent works can be found on Google Scholar.

Selected
Representative
All (in chronological order)

DIFFormer: Scalable (Graph) Transformers Induced by Energy Constrained Diffusion

Qitian Wu, Chenxiao Yang, Wentao Zhao, Yixuan He, David Wipf and Junchi Yan

International Conference on Learning Representations (ICLR) 2023 spotlight presentation, avg. ranking among top 0.5%

Paper Slides Poster Video Code

Summary: We propose a geometric diffusion framework with energy constraints and show its solution aligns with widely used attention networks, upon which we propose diffusion-based Transformers.

NodeFormer: A Scalable Graph Structure Learning Transformer for Node Classification

Qitian Wu, Wentao Zhao, Zenan Li, David Wipf and Junchi Yan

Advances in Neural Information Processing Systems (NeurIPS) 2022 spotlight presentation (less than 5%)

Paper Slides Poster Video Code Blog (Chinese) Blog (English)

Summary: We propose a scalable graph Transformer with efficient all-pair message passing achieved in O(N) complexity. The global attention over 2M nodes only requires 4GB memory.

Handling Distribution Shifts on Graphs: An Invariance Perspective

Qitian Wu, Hengrui Zhang, Junchi Yan and David Wipf

International Conference on Learning Representations (ICLR) 2022

Paper Slides Poster Code

Summary: We formulate out-of-distribution generalization on graphs and discuss how to leverage (causal) invariance principle for handling graph-based distribution shifts.

SGFormer: Simplifying and Empowering Transformers for Large-Graph Representations

Qitian Wu, Wentao Zhao, Chenxiao Yang, Hengrui Zhang, Fan Nie, Haitian Jiang, Yatao Bian and Junchi Yan

Advances in Neural Information Processing Systems (NeurIPS) 2023

Paper Slides Code

Summary: We propose an efficient Transformer that only uses one-layer global attention and significantly reduces computation cost for large-graph representations.

DIFFormer: Scalable (Graph) Transformers Induced by Energy Constrained Diffusion

Qitian Wu, Chenxiao Yang, Wentao Zhao, Yixuan He, David Wipf and Junchi Yan

International Conference on Learning Representations (ICLR) 2023 spotlight presentation, avg. ranking among top 0.5%

Paper Slides Poster Video Code

Summary: We propose a geometric diffusion framework with energy constraints and show its solution aligns with widely used attention networks, upon which we propose diffusion-based Transformers.

Energy-based Out-of-Distribution Detection for Graph Neural Networks

Qitian Wu, Yiting Chen, Chenxiao Yang, and Junchi Yan

International Conference on Learning Representations (ICLR) 2023

Paper Slides Poster Code

Summary: We extract a OOD detection model from GNN classifier through energy-based models and energy-based belief propagation, reducing FPR95 over SOTAs by up to 44.8%.

Graph Neural Networks are Inherently Good Generalizers: Insights by Bridging GNNs and Multi-Layer Perceptrons

Chenxiao Yang, Qitian Wu, Jiahua Wang and Junchi Yan

International Conference on Learning Representations (ICLR) 2023

Paper Code

Summary: We identify that the efficacy of GNNs over MLP mostly stems from its inherent better generalization, demonstrated by experiments on sixteen benchmarks and neural tangent kernel theory.

NodeFormer: A Scalable Graph Structure Learning Transformer for Node Classification

Qitian Wu, Wentao Zhao, Zenan Li, David Wipf and Junchi Yan

Advances in Neural Information Processing Systems (NeurIPS) 2022 spotlight presentation (less than 5%)

Paper Slides Poster Video Code Blog (Chinese) Blog (English)

Summary: We propose a scalable graph Transformer with efficient all-pair message passing achieved in O(N) complexity. The global attention over 2M nodes only requires 4GB memory.

Learning Substructure Invariance for Out-of-Distribution Molecular Representations

Nianzu Yang, Kaipeng Zeng, Qitian Wu, Xiaosong Jia and Junchi Yan

Advances in Neural Information Processing Systems (NeurIPS) 2022 spotlight presentation (less than 5%)

Paper Code

Summary: We propose a invariant learning approach for molecular property prediction under distribution shifts and achieve SOTA results on OGB-mol and DrugOOD benchmarks.

Geometric Knowledge Distillation: Topology Compression for Graph Neural Networks

Chenxiao Yang, Qitian Wu and Junchi Yan

Advances in Neural Information Processing Systems (NeurIPS) 2022

Paper Code

Summary: We explore geometric knowledge distillation, empowered by neural heat kernel, that can generalize the topological knowledge from larger GNNs to smaller one.

Towards Out-of-Distribution Sequential Event Prediction: A Causal Treatment

Chenxiao Yang, Qitian Wu, Qingsong Wen, Zhiqiang Zhou, Liang Sun and Junchi Yan

Advances in Neural Information Processing Systems (NeurIPS) 2022

Paper Code

Summary: We use causal analysis to reveal the limitations of Maximum Likelihood Estimation for sequential prediction under distribution shifts and propose an effective treatment based on backdoor adjustment.

Handling Distribution Shifts on Graphs: An Invariance Perspective

Qitian Wu, Hengrui Zhang, Junchi Yan and David Wipf

International Conference on Learning Representations (ICLR) 2022

Paper Slides Poster Code

Summary: We formulate out-of-distribution generalization on graphs and discuss how to leverage (causal) invariance principle for handling graph-based distribution shifts.

Towards Open-World Feature Extrapolation: An Inductive Graph Learning Approach

Qitian Wu, Chenxiao Yang and Junchi Yan

Advances in Neural Information Processing Systems (NeurIPS) 2021

Paper Slides Poster Code

Summary: We propose a graph representation learning approach for handling feature space expansion from training data to testing data.

From Canonical Correlation Analysis to Self-supervised Graph Neural Networks

Hengrui Zhang, Qitian Wu, Junchi Yan, David Wipf and Philip S. Yu

Advances in Neural Information Processing Systems (NeurIPS) 2021

Paper Code

Summary: We introduce a simple-yet-effective contrastive objective for self-supervised learning on graphs.

Towards Open-World Recommendation: An Inductive Model-based Collaborative Filtering Approach

Qitian Wu, Hengrui Zhang, Xiaofeng Gao, Junchi Yan and Hongyuan Zha

International Conference on Machine Learning (ICML) 2021

Paper Slides Poster Code

Summary: We propose a latent structure inference model to handle new unseen users in the testing phase of recommender systems.

Learning Latent Process from High-Dimensional Event Sequences via Efficient Sampling

Qitian Wu, Zixuan Zhang, Xiaofeng Gao, Junchi Yan and Guihai Chen

Advances in Neural Information Processing Systems (NeurIPS) 2019

Paper Code

Dual Sequential Prediction Models Linking Sequential Recommendation and Information Dissemination

Qitian Wu, Yirui Gao, Xiaofeng Gao, Paul Weng and Guihai Chen

ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) 2019

Paper Code

Summary: We use a duality perspective for unifying two sequential information retrieval problems into one model with mutual enhancement enabled.

Dual Graph Attention Networks for Deep Latent Representation of Multifaceted Social Effects in Recommender Systems

Qitian Wu, Hengrui Zhang, Xiaofeng Gao, Peng He, Paul Weng, Han Gao and Guihai Chen

The Web Conference (WWW) 2019 long oral representation (only 1/2 among all accepted long papers)

Paper Code

Summary: We propose a dual graph attention model that captures the social homophily and influence effects among users and items in recommender systems.

Learning Divergence Fields for Shift-Robust Message Passing

Qitian Wu, Fan Nie, Chenxiao Yang and Junchi Yan

International Conference on Machine Learning (ICML) 2024

How Graph Neural Networks Learn: Lessons from Training Dynamics

Chenxiao Yang, Qitian Wu, David Wipf, Ruoyu Sun and Junchi Yan

International Conference on Machine Learning (ICML) 2024

Graph Out-of-Distribution Detection Goes Neighborhood Shaping

Tianyi Bao, Qitian Wu, Zetian Jiang, Yiting Chen, Jiawei Sun and Junchi Yan

International Conference on Machine Learning (ICML) 2024

Graph Out-of-Distribution Generalization via Causal Intervention

Qitian Wu, Fan Nie, Chenxiao Yang, Tianyi Bao and Junchi Yan

The Web Conference (WWW) 2024 oral presentation

Rethinking Cross-Domain Sequential Recommendation Under Open-World Assumptions

Wujiang Xu, Qitian Wu, Runzhong Wang, Mingming Ha, Qiongxu Ma, Linxun Chen, Bing Han and Junchi Yan

The Web Conference (WWW) 2024

SGFormer: Simplifying and Empowering Transformers for Large-Graph Representations

Qitian Wu, Wentao Zhao, Chenxiao Yang, Hengrui Zhang, Fan Nie, Haitian Jiang, Yatao Bian and Junchi Yan

Advances in Neural Information Processing Systems (NeurIPS) 2023

Unleashing the Power of Graph Data Augmentation on Covariate Distribution Shift

Yongduo Sui, Qitian Wu, Jiancan Wu, Qing Cui, Longfei Li, Jun Zhou, Xiang Wang, Xiangnan He

Advances in Neural Information Processing Systems (NeurIPS) 2023

GraphGlow: Universal and Genralizable Structure Learning for Graph Neural Networks

Wentao Zhao, Qitian Wu, Chenxiao Yang and Junchi Yan

ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) 2023

MoleRec: Combinatorial Drug Recommendation with SubstructureAware Molecular Representation Learning

Nianzu Yang, Kaipeng Zeng, Qitian Wu, Junchi Yan

The Web Conference (WWW) 2023