Scalable Filesystems and I/O
Overview
In modern clusters, I/O is quickly emerging as the main bottleneck
limiting performance. The need for scalable parallel I/O and file systems
is becoming more and more urgent. The performance of network storage
systems is often limited by overhead in the I/O path, such as memory
copying, network access costs, and protocol overhead. High speed networking
architectures such as InfiniBand and Quadrics create opportunities to
address these issues without changing fundamental principles of production
operating systems.
Objectives
- Efficient Design and Implementation of PVFS over InfiniBand
- Design of iSCSI over InfiniBand Exploiting RDMA
- High Performance Design and Implementation of PVFS2 over Quadrics
Description
We have examined the feasibility of leveraging InfiniBand and Quadrics
technology
to improve I/O performance and scalability of cluster file systems.
By taking advantage of user-level networking and Remote Direct Memory Access
(RDMA) we have implemented The Parallel Virtual File System (PVFS)
over InfiniBand and the second generation of Parallel Virtual File
System (PVFS2) over Quadrics.
In addition, we have suggested Unifier to unify file cache
management and communication buffer management for PVFS over InfiniBand.
Moreover, we have exploited InfiniBand RDMA to improve iSCSI. We plan
to extend these research activities to provide high performance storage
solution for InfiniBand clusters.
Ph.D. Disserations (2) | |
|---|---|
| 1 | R. Rajachandrasekar, Designing Scalable And Efficient I/O Middleware for Fault-Resilient High-performance Computing Clusters, Nov 2014 |
| 2 | J. Wu, Communication and Memory Management in Networked Storage Systems, Sep 2004 |