Jianyu Wang

Ph.D. student
Carnegie Mellon University


Pittsburgh, PA 15213
Email: jianyuw1 [at] andrew.cmu.edu
Google Scholar



My name is Jianyu Wang (王健宇 in Chinese). I am a third-year Ph.D. student at Carnegie Mellon University, advised by professor Gauri Joshi and affiliated with Optimization, Probability, and Learning (OPAL) group, Parallel Data Lab. My research has been supported by Qualcomm Ph.D. fellowship.

Before coming to CMU, I received a Bachelors with honor in Electronic Engineering at Tsinghua University, where I worked with professor Yuan Shen on several projects about statistical inference. I was also fortunate to visit professor Yu Bin's group at U.C. Berkeley in summer 2016, working on the optimization landscape of neural networks.

Research Interests

My research focuses on providing fundamental understandings of how distributed training algorithms are influenced by system-level variability in the computing infrastructure, and statistical variability in the training data. Inspired by the theoretical insights, I seek to design system- and data-aware distributed training algorithms that can seamless scale to a large number of computing nodes.

I am interested in distributed machine learning, federated learning, and distributed optimization.

Industrial Experiences

   Google Research
      Research Intern, Summer 2020
      Mentor(s): Zachary Garrett, Zheng Xu
      Project: Federated Learning

   Facebook AI Research
      Research Intern, Summer 2019
      Mentor(s): Michael Rabbat, Nicolas Ballas
      Project: Communication-efficient Distributed Deep Learning


   Carnegie Mellon University
      Sept. 2017 -- Present
      Ph.D. student in Electrical & Computer Engineering
      CGPA: 4.0/4.0

   Tsinghua University
      Aug. 2013 -- July. 2017
      B.Eng. in Elelctronic Engineering
      CGPA: 91/100, ranking: top 10%

Publication and Preprints

SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentum
Jianyu Wang, Vinayak Tantia, Nicolas Ballas, Michael Rabbat
ICLR 2020: International Conference on Learning Representations
paper / slides
Lookahead Converges to Stationary Points of Smooth Non-Convex Functions
Jianyu Wang, Vinayak Tantia, Nicolas Ballas, Michael Rabbat
ICASSP 2020: 45th International Conference on Acoustics, Speech, and Signal Processing
paper / slides
Overlap Local-SGD: An Algorithmic Approach to Hide Communication Delays in Distributed SGD
Jianyu Wang, Hao Liang, Gauri Joshi
ICASSP 2020: 45th International Conference on Acoustics, Speech, and Signal Processing
paper / slides / code
Machine Learning on Volatile Instances
Xiaoxi Zhang, Jianyu Wang, Gauri Joshi, Carlee Joe-Wong
INFOCOM 2020: IEEE International Conference on Computer Communications
Slow and Stale Gradients Can Win the Race
Sanghamitra Dutta, Jianyu Wang, Gauri Joshi
Submitted to Journal of Machine Learning Research
MATCHA: Speeding up Decentralized SGD via Matching Decomposition Sampling
Jianyu Wang, Anit Kumar Sahu, Zhouyi Yang, Gauri Joshi, Soummya Kar
Abridged in NeurIPS 2019 Workshop on Federated Learning (Oral & Best Student Paper Award).
paper / slides / code
Adaptive Communication Strategies to Achieve the Best Error-Runtime Trade-off in Local-Update SGD
Jianyu Wang, Gauri Joshi
MLSys 2019 (17% acceptance rate): The Conference on Machine Learning and Systems (Oral)
Abridged in NeurIPS 2018 Workshop on Systems for ML.
paper / slides / video
Cooperative SGD: A Unified Framework for the Design and Analysis of Local-Update Distributed SGD Algorithms
Jianyu Wang, Gauri Joshi
Abridged in ICML 2019 Workshop on Coding Theory for Large-Scale ML (Oral).
paper / slides / video
On the Discretization Schemes in Map-Aided Indoor Localization
Jianyu Wang, Yuan Shen
IEEE Communications Letters / paper

Selected Honors and Awards

Year Honor Description
2019 Distinguished Student Paper Award Workshop on Federated Learning for Data Privacy and Confidentiality (@ NeurIPS 2019)
2019 Qualcomm Innovation Fellowship
2018 Benjamin Garver Lamme/Westinghouse Graduate Fellowship, CMU
2017 College of Engineering Dean’s Fellowship, CMU
2017 Excellent Graduate Awards, Tsinghua Awarded to top 10% among all 3000 undergraduates
2015 Fellowship of Spark Talents Program, Tsinghua Awarded to top 50 students, dedicated to scientific and technological innovations
2012 National Training Team for International Physics Olympiad Selected top 60 students in China
Year Scholorship Description
2016 Aviation Industry of China Scholarship For academic and reasearch excellence
2016 Huawei Scholarship For comprehensive excellence
2015 TP-Link Scholarship For academic, research and social work excellence
2014 Zheng Geru Scholarship For academic excellence

Graduated Courses

Machine Learning

Computer Systems