Zekun Wang

Hi, my name is Zekun Wang (汪泽堃 in Chinese). I am a Researcher in Qwen @ Alibaba Group. I earned my PhD degree from SCIR @ Harbin Institute of Technology.

My research interests have evolved to focus on developing:

• Model/Data-Efficiency & Acceleration: Efficient architecture for XFMRs or hybrid ones; Pruning, distillation, quantization etc. to reduce model size and speedup inference; Efficient training LLMs/MLLMs with a small cost (time or data).

• Multi-modal Models and Applications: Large multi-modal models (comprehensive, generation, or unify the both), which support diverse tasks and can be applied as agents in digital or embodied environments.

I am particularly interested in exploring the intrinsic relationship between model (efficient) architecture and its performance, as well as how to unlock its potential in real-world scenarios.

News:

• 🎉 Two papers have been accepted by NeurIPS 2025. Gated attention is selected as an oral presentation!
• ⚡️ Our new Qwen3-Next models are out! Read the blog 📄, try the chat 💬, and get the 🤗 model. –>

Selected Publications:

Full publications can be found in [Semantic Scholar] [Google Scholar] (* denotes equal contributions.† indicates corresponding author)

• Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free Zihan Qiu*, Zekun Wang*, Bo Zheng*, Zeyu Huang*, Kaiyue Wen, Songlin Yang, Rui Men, Le Yu, Fei Huang, Suozhi Huang, Dayiheng Liu, Jingren Zhou, Junyang Lin
  NeurIPS 2025 Oral
  [paper] [models] [code]
  

• Qwen3-Next: Towards Ultimate Training & Inference Efficiency
  Core Contributor
  [blog] [🤗 model]
  

• Qwen3 Technical Report
  Core Contributor
  Technical Report
  [paper] [collection]
  

• EffiVLM-Bench: A Comprehensive Benchmark for Evaluating Training-Free Acceleration in Large Visual-Languge Models
  Zekun Wang*, MingHua Ma*, Zexin Wang*, Rongchuan Mu*, Ming Liu, Bing Qin
  ACL 2025
  [paper] [project]
  

• Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of Expert Models
  Zihan Qiu, Zeyu Huang, Bo Zheng, Kaiyue Wen, Zekun Wang, Rui Men, Ivan Titov, Dayiheng Liu, Jingren Zhou, Junyang Lin
  ACL 2025
  [pdf]
  

• Aguvis: Unified Pure Vision Agents for Autonomous GUI Interaction
  Yiheng Xu*, Zekun Wang*, Junli Wang*, Dunjie Lu, Tianbao Xie, Amrita Saha, Doyen Sahoo, Tao Yu, Caiming Xiong
  ICML 2025
  [pdf] [project]
  

• Improved Diffusion-based Generative Model with Better Adversarial Robustness
  Zekun Wang*, Mingyang Yi*, Shuchen Xue, Zhenguo Li, Ming Liu, Bing Qin, Zhi-Ming Ma
  ICLR 2025
  [pdf]
  

• AgentTrek: Agent Trajectory Synthesis via Guiding Replay with Web Tutorials
  Yiheng Xu, Dunjie Lu, Zhennan Shen, Junli Wang, Zekun Wang, Yuchen Mao, Caiming Xiong, Tao Yu
  ICLR 2025 Spotlight (Top 5%)
  [pdf] [project]
  

• CFSP: An Efficient Structured Pruning Framework for LLMs with Coarse-to-Fine Activation Information
  Yuxin Wang*, Minghua Ma*, Zekun Wang*†, Jingchang Chen, Huiming Fan, Liping Shan, Qing Yang, Dongliang Xu, Ming Liu, Bing Qin.
  COLING 2025
  [pdf] [project]
  

• Qwen2.5 Technical Report
  Qwen Team
  Technical Report
  [pdf] [project] [collection]
  

• Divide-and-Conquer Meets Consensus: Unleashing the Power of Functions in Code Generation
  Jingchang Chen, Hongxuan Tang, Zheng Chu, Qianglong Chen, Zekun Wang, Ming Liu, Bing Qin
  NeurIPS 2024 Oral
  [pdf]
  

• SmartTrim: Adaptive Tokens and Attention Pruning for Efficient Vision-Language Models
  Zekun Wang*, Jingchang Chen*, Wangchunshu Zhou, Haichao Zhu, Jiafeng Liang, Liping Shan, Ming Liu, Dongliang Xu, Qing Yang, Bing Qin
  COLING 2024 Oral
  [pdf]
  

• Distilled Dual-Encoder Model for Vision-Language Understanding
  Zekun Wang, Wenhui Wang, Haichao Zhu, Ming Liu, Bing Qin, Furu Wei
  EMNLP 2022 [pdf] [project]

• Less Is More: Domain Adaptation with Lottery Ticket for Reading Comprehension
  Haichao Zhu, Zekun Wang, Heng Zhang, Ming Liu, Sendong Zhao, Bing Qin
  Findings of EMNLP 2021
  [pdf]
  

Powered by Jekyll and Minimal Light theme.