Determining the optimal protocols for transfer of large data files

Other

2015

Permanent link to this Item
Authors
Journal Title
Link to Journal
Journal ISSN
Volume Title
Publisher
Publisher

University of Cape Town

License
Series
Abstract
The transfer of large data files remains a necessity for scientific research, including Bioinformatics, despite alternate solutions like cloud computing. When transferring large files, there are many transfer protocols to choose from. In this paper, a lightweight testbed is developed and used to test the SCP, FTP, HPN-SCP and GridFTP protocols for transfer of large files between the Universities of Cape Town and the Western Cape. A comparison of GridFTP’s ‘UDT’ mode (built on UDP) and its default mode is also done using the same testbed. We investigate the effect of a congested network on transfers. Speed and data efficiency are compared on a stable network to determine if any one protocol is significantly better than the rest. We find that for network speeds of around 200Mbit/s, the protocols achieve very similar speeds. The TCP based protocols achieved very similar efficiency, while GridFTP in UDT mode had much higher overhead.
Description

Reference:

Collections