Date: 07/12/2010

Moving large data sets across the Internet is becoming more and more feasible.  For example, moving 1 Terabyte (TB) of data between most large research institutions will only take around 8 hours. This assumes a 1 Gbps or higher end-to-end network data path, and that only 1/3 of the capacity is used, leaving room for other network traffic.

According to the Energy Sciences Network (ESnet), the expected time to copy 1 TB of data:

  • 30.00 hrs – using a 100 Mbps network.
  •   3.00 hrs – using a 1 Gbps network.  Fast disk drives are needed.
  •     .33 hrs – using a 10 Gbps network.  Requires extremely fast disk drives and file system.

This is possible due to University of Iowa’s access to high-speed networks, e.g., BOREAS, CIC OmniPoP, and Internet2

Currently, the University of Iowa has (1) 10 Gbps circuit, soon to be (2) 10 Gbps circuits, for the campus primary external network connectivity. Additionally, the campus has:

  • Direct peering to members of the CIC OmniPoP.
  • A share of (2) 10 Gbps connections to Internet2 for applications in research and education as well as connections to Internet2-managed peering to commercial Internet Service Providers.
  • 1 Gbps of commercial Internet access.

ESnet provides an excellent online resource that details the issues to consider when sending large files over the Internet:

  • TCP/IP performance tuning
  • Expected throughput
  • File transfer tools
  • Network performance measurement tools

To learn more about transferring large data sets over the Internet, please visit the ESnet Network Performance Knowledge Base.

Article number: 
1605
Last updated: 
May 19, 2016