The International Conference for High Performance Computing, Networking, Storage and Analysis
UDT Based Application Performance for High Speed Data Transfer.
Student: Joshua Miller (University of Chicago)
Supervisor: Allison Heath (Open Science Data Cloud)
Abstract: Over high performance wide area networks, TCP has been shown to reach a data transfer bottleneck before UDP. UDT is a reliable UDP based application level protocol designed to circumvent this limit. Despite increased data transfer speeds, the lack of UDT based file transfer tools has prevented its wide spread use within the scientific community.
Our purpose has been to create and test UDT based applications for bulk data transfer. This suite (UDStar) includes a wrapper for the stand alone file transfer tools rsync and scp as well as a UDT based netcat tool. Inclusion as a prefix to standard rsync and scp commands, UDStar minimizes the modification of existing work flows to aid its adoption into the scientific community.
Currently, UDStar is used to sync NASA satellite and ENCODE data sets. This poster presents the performance increases for transfers over 10G wide area networks.