In modern clusters, I/O is quickly emerging as the main bottleneck
limiting performance. The need for scalable parallel I/O and file systems
is becoming more and more urgent. The performance of network storage
systems is often limited by overhead in the I/O path, such as memory
copying, network access costs, and protocol overhead. High speed networking
architectures such as InfiniBand and Quadrics create opportunities to
address these issues without changing fundamental principles of production
We have examined the feasibility of leveraging InfiniBand and Quadrics
to improve I/O performance and scalability of cluster file systems.
By taking advantage of user-level networking and Remote Direct Memory Access
(RDMA) we have implemented The Parallel Virtual File System (PVFS)
over InfiniBand and the second generation of Parallel Virtual File
System (PVFS2) over Quadrics.
In addition, we have suggested Unifier to unify file cache
management and communication buffer management for PVFS over InfiniBand.
Moreover, we have exploited InfiniBand RDMA to improve iSCSI. We plan
to extend these research activities to provide high performance storage
solution for InfiniBand clusters.
Broad research issues being investigated in this direction:
Efficient Design and Implementation of PVFS over InfiniBand
Design of iSCSI over InfiniBand Exploiting RDMA
High Performance Design and Implementation of PVFS2 over Quadrics
R. Noronha, and D. K. Panda,
IMCa: A High Performance Caching Frontend for GlusterFS on InfiniBand.
Int'l Conference on Parallel Processing, Portland, OR, 2008.