High Performance Networking for TCP-based Applications

The Transmission Control Protocol (TCP) is one of the universally accepted transport layer protocols in today's networking world. The introduction of gigabit speed networks a few years back had challenged the traditional TCP/IP implementation in two aspects, namely performance and CPU requirements. In order to allow TCP/IP based applications achieve the performance provided by these networks while demanding lesser CPU resources, researchers came up with solutions in two broad directions: user-level sockets and TCP Offload Engines. Both these approaches concentrate on optimizing the protocol stack either by replacing the TCP stack with zero-copy, OS-bypass protocols such as VIA, EMP or by offloading the entire or part of the TCP stack on to hardware. However, these offloading techniques have not been completely successful in avoiding the multiple copies and kernel context switches, mainly due to certain fundamental bottlenecks in the traditional TCP implementation including the memory traffic associated with the data streaming semantics, buffering for out-of-order packets, and several others.

As a part of this research, we have taken up the challenge of identifying these bottlenecks and implementing rectifying algorithms. Our approach to this problem involves the development of the Direct Data Placement (DDP) and the Marker PDU Aligned (MPA) protocols, where the data is directly placed into the user buffer, thus avoiding the intermediate copy and buffering of data. The challenges in this project include the message segmentation algorithm and the data re-assembly queue. To support the applications which are designed to take advantage of the data-streaming capabilities of TCP, we have retained the current sockets implementation, giving the user an option to choose either of them.

Current research along this direction investigates the following issues:

• Design and Implementation of DDP and MPA over 10-GigE
• Analysis of High Performance User-Level Sockets over InfiniBand
• Efficient Protocol Offload Engines on Programmable NIC

Publications:

Journals

• H. -W. Jin, P. Balaji, C. Yoo, J. -Y. Choi, and D. K. Panda, Exploiting NIC Architectural Support for Enhancing IP based Protocols on High Performance Networks, Journal of Parallel and Distributed Computing, Special Issue on Design and Performance of Networks for Super-, Cluster-, and Grid-Computing, in press.

Conferences/Workshops

• P. Balaji, S. Bhagvat, H. -W. Jin, D. K. Panda Asynchronous Zero-copy Communication for Synchronous Sockets in the Sockets Direct Protocol (SDP) over InfiniBand , Communication Architecture for Clusters (CAC).
• V. Vishwanath, P. Balaji, W. Feng, J. Leigh and D. K. Panda A Case for UDP Offload Engines in LambdaGrids , Workshop on Protocols for Fast Long Distance Networks (PFLDNet), Feb, 2006, Nara, Japan.
• P. Balaji, H. -W. Jin, K. Vaidyanathan and D. K. Panda, Supporting iWARP Compatibility and Features for Regular Network Adapters, Workshop on Remote Direct Memory Access (RDMA): Applications, Implementations, and Technologies (RAIT 2005), Sept. 2005, in conjunction with the IEEE Cluster 2005. Conference Slides
• P. Balaji, W. Feng, Q. Gao, R. Noronha, W. Yu and D. K. Panda, Head-to-TOE Evaluation of High-Performance Sockets over Protocol Offload Engines, IEEE Cluster Computing 2005, Sept. 2005. Conference Slides
• H. -W. Jin, S. Narravula, G. Brown, K. Vaidyanathan, P. Balaji, and D. K. Panda, Performance Evaluation of RDMA over IP: A Case Study with the Ammasso Gigabit Ethernet NIC, Proceedings of Workshop on High Performance Interconnects for Distributed Computing (HPI-DC); In conjunction with HPDC-14, Research Triangle Park, NC, USA, July 2005. Conference Slides
• W. Feng, P. Balaji, C. Baron, L. N. Bhuyan, and D. K. Panda Performance Characterization of a 10-Gigabit Ethernet TOE, Hot Interconnect (HOTI 05), August, 2005. Conference Slides
• P. Balaji, H. V. Shah and D. K. Panda, Sockets vs RDMA Interface over 10-Gigabit Networks: An In-depth analysis of the Memory Traffic Bottleneck, Workshop on Remote Direct Memory Access (RDMA): Applications, Implementations, and Technologies (RAIT 2004). Sept 20th, 2004, in conjunction with the IEEE Cluster 2004, San Diego, California.
• P. Balaji, J. Wu, T. Kurc, U. Catalyurek, D. K. Panda and J. Saltz, Impact of High Performance Sockets on Data Intensive Applications, IEEE International Symposium on High Performance Distributed Computing (HPDC) 2003, Seattle, WA.
  
• P. Balaji, P. Shivam, P. Wyckoff and D. K. Panda, High Performance User-Level Sockets over Gigabit Ethernet, IEEE International Conference on Cluster Computing, Sept 23-26, 2002, Chicago, IL.
  

Technical Reports

• P. Balaji, W. Feng and D. K. Panda The Convergence of Ethernet and Ethernot: A 10-Gigabit Ethernet Perspective, OSU-CISRC-1/06-TR10.
• H. -W. Jin, S. Narravula, G. Brown, K. Vaidyanathan, P. Balaji, and D. K. Panda, Performance Evaluation of RDMA over IP: A Case Study with Ammasso Gigabit Ethernet NIC, OSU-CISRC-6/05-TR40.
• H. -W. Jin, P. Balaji, C. Yoo, J. -Y. Choi, and D. K. Panda, Exploiting NIC Architectural Support for Enhancing IP based Protocols on High Performance Networks, OSU-CISRC-5/04-TR37.

Graduate Student Researchers: