CONVOHPC 2024

1:30 - 1:35

Opening Remarks

Hari Subramoni and Aamir Shafi The Ohio State University

1:35 - 2:30

Speaker: Arvind Ramanathan, Argonne National Laboratory

Session Chair: Hari Subramoni, The Ohio State University

Title: The Decade Ahead: Building Frontier AI Systems for Science and the Path to Zettascale

Abstract: The successful development of transformative applications of AI for science, medicine and energy research will have a profound impact on the world. The rate of development of AI capabilities continues to accelerate, and the scientific community is becoming increasingly agile in using AI, leading to us to anticipate significant changes in how science and engineering goals will be pursued in the future. Frontier AI (the leading edge of AI systems) enables small teams to conduct increasingly complex investigations, accelerating some tasks such as generating hypotheses, writing code, or automating entire scientific campaigns. However, certain challenges remain resistant to AI acceleration such as human-to-human communication, large-scale systems integration, and assessing creative contributions. Taken together these developments signify a shift toward more capital-intensive science, as productivity gains from AI will drive resource allocations to groups that can effectively leverage AI into scientific outputs, while other will lag. In addition, with AI becoming the major driver of innovation in high-performance computing, we also expect major shifts in the computing marketplace over the next decade, we see a growing performance gap between systems designed for traditional scientific computing vs those optimized for large-scale AI such as Large Language Models. In part, as a response to these trends, but also in recognition of the role of government supported research to shape the future research landscape the U. S. Department of Energy has created the FASST (Frontier AI for Science, Security and Technology) initiative. FASST is a decadal research and infrastructure development initiative aimed at accelerating the creation and deployment of frontier AI systems for science, energy research, national security. I will review the goals of FASST and how we imagine it transforming the research at the national laboratories. Along with FASST, I’ll discuss the goals of the recently established Trillion Parameter Consortium (TPC), whose aim is to foster a community wide effort to accelerate the creation of large-scale generative AI for science. Additionally, I'll introduce the AuroraGPT project an international collaboration to build a series of multilingual multimodal foundation models for science, that are pretrained on deep domain knowledge to enable them to play key roles in future scientific enterprises.

Speaker Bio: Arvind Ramanathan is a computational biologist in the Data Science and Learning Division at Argonne National Laboratory and a senior scientist at the University of Chicago Consortium for Advanced Science and Engineering (CASE). His research interests are at the intersection of data science, high performance computing and biological/biomedical sciences.

His research focuses on three areas focusing on scalable statistical inference techniques: (1) for analysis and development of adaptive multi-scale molecular simulations for studying complex biological phenomena (such as how intrinsically disordered proteins self assemble, or how small molecules modulate disordered protein ensembles), (2) to integrate complex data for public health dynamics, and (3) for guiding design of CRISPR-Cas9 probes to modify microbial function(s).

He has published over 30 papers, and his work has been highlighted in the popular media, including NPR and NBC News. He obtained his Ph.D. in computational biology from Carnegie Mellon University, and was the team lead for integrative systems biology team within the Computational Science, Engineering and Division at Oak Ridge National Laboratory. More information about his group and research interests can be found at http://​ramanathanlab​.org.

2:30 - 3:00

Break

3:00-3:30

Speaker: Abhinav Bhatele, University of Maryland

Session Chair: Hari Subramoni, The Ohio State University

Title: The Role of Generative AI in HPC Code Development

Abstract:

Artificial intelligence (AI) and large language models (LLMs) specifically, have recently been used to model source code, which has proven to be effective for a variety of software development tasks such as code completion, summarization, translation, and debugging, among others. We are witnessing an increasing use of AI tools such as ChatGPT and GitHub CoPilot as assistants in code development. However, it is unclear how well such LLM-powered technologies work for parallel code development. Writing, debugging and optimizing parallel code is hard, and the question before the HPC community is -- does Generative AI hold the potential for revolutionizing HPC software development. In this talk, I will address the shortcomings of current LLMs when used for parallel code development and how we can close the gap toward building HPC-capable LLMs. I will further highlight emerging areas of research such as improving code modeling capabilities to facilitate various aspects of HPC code development, such as generating correct and efficient parallel code, reasoning about parallel performance, and much more.

Speaker Bio:Abhinav Bhatele is an associate professor in the department of computer science, and director of the Parallel Software and Systems Group at the University of Maryland, College Park. His research interests are broadly in systems and AI, with a focus on parallel computing and distributed AI. He has published research in parallel programming models and runtimes, network design and simulation, applications of machine learning to parallel systems, parallel deep learning, and on analyzing/visualizing, modeling and optimizing the performance of parallel software and systems. Abhinav has received best paper awards at Euro-Par 2009, IPDPS 2013, IPDPS 2016, and PDP 2024, and a best poster award at SC 2023. He was selected as a recipient of the IEEE TCSC Award for Excellence in Scalable Computing (Early Career) in 2014, the LLNL Early and Mid-Career Recognition award in 2018, the NSF CAREER award in 2021, the IEEE TCSC Award for Excellence in Scalable Computing (Middle Career) in 2023, and the UIUC CS Early Career Academic Achievement Alumni Award in 2024.

Abhinav received a B.Tech. degree in Computer Science and Engineering from I.I.T. Kanpur, India in May 2005, and M.S. and Ph.D. degrees in Computer Science from the University of Illinois at Urbana-Champaign in 2007 and 2010 respectively. He was a post-doc and later computer scientist in the Center for Applied Scientific Computing at Lawrence Livermore National Laboratory from 2011-2019. Abhinav is an associate editor of the IEEE Transactions on Parallel and Distributed Systems (TPDS). He was one of the General Chairs of IEEE Cluster 2022, and Research Papers Chair of ISC 2023.

3:30-4:00

Speaker: Murali Krishna Emani, Argonne National Laboratory

Session Chair: Hari Subramoni, The Ohio State University

Title: LM4HPC: Towards Effective Language Model Application in High-Performance Computing

Abstract: In recent years, language models (LMs),such as GPT-4,have been widely used in multiple domains, including natural language processing, visualization, and so on. However, applying them for analyzing and optimizing high-performance computing (HPC) software is still challenging due to the lack of HPC-specific support. In this talk, we present the design of the LM4HPC framework to facilitate the research and development of HPC software analyses and optimizations using LMs. Tailored for supporting HPC datasets, AI models, and pipelines, our framework is built on top of a range of components from different levels of the machine learning software stack, with Hugging Face-compatible APIs. Using three representative tasks, we evaluated the prototype of our framework. The results show that LM4HPC can help users quickly evaluate a set of state-of-the-art models and generate insightful leaderboards.

Speaker Bio: Murali Emani is a Computer Scientist in the AI/ML group with the Argonne Leadership Computing Facility (ALCF). His research interests are in Scalable machine learning, AI accelerators, high-performance computing, and performance optimization. Murali co-leads the ALCF AI Testbed to explore the performance, efficiency of novel AI accelerators for scientific machine learning applications. He has published papers in esteemed venues such as ATC, SIGMETRICS, SC, HPDC, and PLDI. He also co-chairs MLPerf HPC group at MLCommons to benchmark large scale ML on HPC systems.

4:00-4:30

Speaker: Brandon Biggs, Idaho National Lab

Session Chair: Hari Subramoni, The Ohio State University

Title: Outcomes of HPC User Support using a Science Gateway AI Assistant

Abstract: High Performance Computing (HPC) is a vital resource for nuclear energy research, facilitating advanced simulations and complex modeling of the quantification and qualification of advanced reactor technology. However, a common gap in knowledge exists around utilizing HPC systems, particularly for nuclear energy researchers unfamiliar with specific systems. A researcher may be well-versed in using one HPC system and understanding its associated processes. Yet, they might struggle when faced with a different HPC system and its unique processes. HPC support staff play a crucial role in addressing these challenges by providing educational resources and assisting users. However, they also face the challenge of maintaining these systems and ensuring they run efficiently for all users, a responsibility that can be challenging to scale effectively with the increasing demand and expansion of HPC systems. This paper addresses this knowledge gap with an artificial intelligence (AI) assistant that offers on-demand, site-specific HPC support for researchers. Idaho National Laboratory (INL) has deployed an AI assistant that is intended to supplement expert HPC support staff and assist nuclear energy researchers. This paper reports on a 4 1/2-month study evaluating the integration of an AI assistant within a science gateway, with the goal of enhancing existing HPC support.

Speaker Bio: Brandon Biggs is a High-Performance Computing (HPC) Systems Administrator in Idaho National Laboratory’s Advanced Scientific Computing Department, part of the Nuclear Science & Technology directorate. He has experience with HPC system management, software integration, web development, and machine learning. Before joining INL in 2019 he was a systems administrator for the College of Science and Engineering at Idaho State University, managing scientific data and computational systems, as well as educational computing resources for students. He also worked as a Graduate Research Assistant and Computer Science Outreach Coordinator while at Idaho State. He is nearing completion for a M.S in computer science from Idaho State University and earned his B.S in computer science from Idaho State University in 2018.

4:30-5:00

Speaker: Pouya Kousha, NVIDIA

Session Chair: Hari Subramoni, The Ohio State University

Title: KG-LLM: A Novel LLM-enabled Framework for Accelerating the Creation of Knowledge Graphs for HPC

Abstract:

Speaker Bio:Pouya Kousha is an HPC Performance Engineer at NVIDIA. He received his PhD at The Ohio State University, USA, contributing to the Nowlab research group led by Prof. DK. Panda and Dr. H. Subramoni. His research targets real-time analysis, monitoring, and profiling of High-Performance Computing (HPC) systems, with his thesis centered on developing AI-enabled tools for HPC communication analysis. He was the lead developer for OSU INAM, a tool for HPC cluster monitoring. With interests in utilizing AI for HPC, conversational AI, distributed systems, and profiling tools, he has reviewed for over 20 HPC conferences and have been a program committee member of various HPC conferences.

Closing Remarks

Hari Subramoni and Aamir Shafi The Ohio State University