NOWLAB :: Publications

Journals
Conferences &
Workshops

Journals (1)
1	Q. Anthony, B. Michalowicz, J. Hatef, L. Xu, M. Abduljabbar, A. Shafi, H. Subramoni, and DK Panda, Understanding and Characterizing Communication Characteristics for Distributed Transformer Models, IEEE Micro, Jan 2025.

Conferences & Workshops (20)
1	Scaling Large Language Model Training on Frontier with Low-Bandwidth Partitioning L. Xu, Q. Anthony, J. Hatef, A. Shafi, H. Subramoni, and DK Panda, 31st IEEE International Conference on High Performance Computing, Data, and Analytics, Dec 2024 [Bib - Plain]
2	Demystifying the Communication Characteristics for Distributed Transformer Models Q. Anthony, B. Michalowicz, J. Hatef, L. Xu, M. Abduljabbar, A. Shafi, H. Subramoni, and DK Panda, IEEE Hot Interconnects Symposium 2024, Aug 2024 [Q. Anthony and B. Michalowicz are co-lead authors] [Bib - Plain]
3	The Case for Co-Designing Model Architectures with Hardware Q. Anthony, J. Hatef, Deepak Narayanan, Stella Biderman, Stas Bekman, Junqi Yin, A. Shafi, H. Subramoni, and DK Panda, 53rd International Conference on Parallel Processing, Aug 2024 [Bib - Plain]
4	Infer-HiRes: Accelerating Inference for High-Resolution Images with Quantization and Distributed Deep Learning R. Gulhane, Q. Anthony, H. Subramoni, and DK Panda, Practice and Experience in Advanced Research Computing, Jul 2024 [Jul 2024] [Bib - Plain]
5	Exploiting Inter-Layer Expert Affinity for Accelerating Mixture-of-Experts Model Inference J. Yao, Q. Anthony, A. Shafi, H. Subramoni, and DK Panda, 38th IEEE International Parallel & Distributed Processing Symposium, May 2024 [Bib - Plain]
6	Accelerating Large Language Model Training with Hybrid GPU-based Compression L. Xu, Q. Anthony, Q. Zhou, N. Alnaasan, R. Gulhane, A. Shafi, H. Subramoni, and DK Panda, IEEE/ACM International Symposium on Cluster, Cloud, and Internet Computing 2024, May 2024 [Bib - Plain]
7	MCR-DL: Mix-and-Match Communication Runtime for Deep Learning Q. Anthony, Ammar Awan, J. Rasley, Y. He, A. Shafi, M. Abduljabbar, H. Subramoni, and DK Panda, 37th IEEE International Parallel & Distributed Processing Symposium (IPDPS '23), May 2023 [Bib - Plain]
8	Accelerating Distributed Deep Learning Training with Compression Assisted Allgather and Reduce-Scatter Communication Q. Zhou, Q. Anthony, L. Xu, A. Shafi, M. Abduljabbar, H. Subramoni, and DK Panda, 37th IEEE International Parallel & Distributed Processing Symposium (IPDPS '23), May 2023 [Bib - Plain]
9	ScaMP: Scalable Meta-Parallelism for Deep Learning Search Q. Anthony, L. Xu, A. Shafi, H. Subramoni, and DK Panda, THE 23RD IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, May 2023 [Bib - Plain]
10	Accelerating Broadcast Communication with GPU Compression for Deep Learning Workloads Q. Zhou, Q. Anthony, A. Shafi, H. Subramoni, and DK Panda, 29th IEEE International Conference on High Performance Computing, Data, and Analytics, Dec 2022 [Bib - Plain]
11	Highly Efficient Alltoall and Alltoallv Communication Algorithms for GPU Systems C. Chen, K. Khorassani, Q. Anthony, A. Shafi, H. Subramoni, and DK Panda, Heterogeneity in Computing Workshop (HCW 2022), May 2022 [held in conjunction with IPDPS'22] [Bib - Plain]
12	Accelerating MPI All-to-All Communication with Online Compression on Modern GPU Clusters Q. Zhou, P. Kousha, Q. Anthony, K. Khorassani, A. Shafi, H. Subramoni, and DK Panda, ISC HIGH PERFORMANCE 2022, May 2022 [Bib - Plain]
13	Hy-Fi: Hybrid Five-Dimensional Parallel DNN Training on High-Performance GPU Clusters A. Jain, A. Shafi, Q. Anthony, P. Kousha, H. Subramoni, and DK Panda, ISC HIGH PERFORMANCE 2022, May 2022 [Bib - Plain]
14	Scaling Single-Image Super-Resolution Training on Modern HPC Clusters: Early Experiences Q. Anthony, L. Xu, H. Subramoni, and DK Panda, Scalable Deep Learning over Parallel And Distributed Infrastructures, May 2021 [Bib - Plain]
15	Adaptive and Hierarchical Large Message All-to-all Communication Algorithms for Large-scale Dense GPU Systems K. Khorassani, C. Chu, Q. Anthony, H. Subramoni, and DK Panda, The 21st IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, May 2021 [Bib - Plain]
16	GEMS: GPU Enabled Memory Aware Model Parallelism System for Distributed DNN Training A. Jain, Ammar Awan, A. Aljuhani, J. Hashmi, Q. Anthony, H. Subramoni, DK Panda, R. Machiraju, and A. Parwani, SC 2020, Nov 2020 [Bib - Plain]
17	Accelerating GPU-based Machine Learning in Python using MPI Library: A Case Study with MVAPICH2-GDR M. Ghazimirsaeed, Q. Anthony, A. Shafi, H. Subramoni, and DK Panda, 6th Workshop on Machine Learning in HPC Environments, Nov 2020 [Bib - Plain]
18	HyPar-Flow: Exploiting MPI and Keras for Scalable Hybrid-Parallel DNN Training with TensorFlow Ammar Awan, A. Jain, Q. Anthony, H. Subramoni, and DK Panda, ISC HIGH PERFORMANCE 2020, Jun 2020 [Bib - Plain]
19	Efficient Training of Semantic Image Segmentation on Summit using Horovod and MVAPICH2-GDR Q. Anthony, Ammar Awan, A. Jain, H. Subramoni, and DK Panda, Scalable Deep Learning over Parallel and Distributed Infrastructures (ScaDL) at IPDPS '20, May 2020 [Bib - Plain]
20	Performance Characterization of DNN Training using TensorFlow and PyTorch on Modern Clusters A. Jain, Ammar Awan, Q. Anthony, H. Subramoni, and DK Panda, 21st IEEE International Conference on Cluster Computing, Sep 2019 [Bib - Plain]

NOWLAB: Network Based Computing Lab

This page lists the publications by Quentin Anthony

Journals (1)

Conferences & Workshops (20)

Scaling Large Language Model Training on Frontier with Low-Bandwidth Partitioning

Demystifying the Communication Characteristics for Distributed Transformer Models

The Case for Co-Designing Model Architectures with Hardware

Infer-HiRes: Accelerating Inference for High-Resolution Images with Quantization and Distributed Deep Learning

Exploiting Inter-Layer Expert Affinity for Accelerating Mixture-of-Experts Model Inference

Accelerating Large Language Model Training with Hybrid GPU-based Compression

MCR-DL: Mix-and-Match Communication Runtime for Deep Learning

Accelerating Distributed Deep Learning Training with Compression Assisted Allgather and Reduce-Scatter Communication

ScaMP: Scalable Meta-Parallelism for Deep Learning Search

Accelerating Broadcast Communication with GPU Compression for Deep Learning Workloads

Highly Efficient Alltoall and Alltoallv Communication Algorithms for GPU Systems

Accelerating MPI All-to-All Communication with Online Compression on Modern GPU Clusters

Hy-Fi: Hybrid Five-Dimensional Parallel DNN Training on High-Performance GPU Clusters

Scaling Single-Image Super-Resolution Training on Modern HPC Clusters: Early Experiences

Adaptive and Hierarchical Large Message All-to-all Communication Algorithms for Large-scale Dense GPU Systems

GEMS: GPU Enabled Memory Aware Model Parallelism System for Distributed DNN Training

Accelerating GPU-based Machine Learning in Python using MPI Library: A Case Study with MVAPICH2-GDR

HyPar-Flow: Exploiting MPI and Keras for Scalable Hybrid-Parallel DNN Training with TensorFlow

Efficient Training of Semantic Image Segmentation on Summit using Horovod and MVAPICH2-GDR

Performance Characterization of DNN Training using TensorFlow and PyTorch on Modern Clusters