Moving large data sets across the Internet is becoming more and more feasible. For example, moving 1 Terabyte (TB) of data between most large research institutions will only take around 8 hours. This assumes a 1 Gbps or higher end-to-end network data path, and that only 1/3 of the capacity is used, leaving room for other network traffic.
According to the Energy Sciences Network (ESnet), the expected time to copy 1 TB of data:
- 30.00 hrs – using a 100 Mbps network.
- 3.00 hrs – using a 1 Gbps network. Fast disk drives are needed.
- .33 hrs – using a 10 Gbps network. Requires extremely fast disk drives and file system.
Currently, the University of Iowa has (1) 10 Gbps circuit, soon to be (2) 10 Gbps circuits, for the campus primary external network connectivity. Additionally, the campus has:
- Direct peering to members of the CIC OmniPoP.
- A share of (2) 10 Gbps connections to Internet2 for applications in research and education as well as connections to Internet2-managed peering to commercial Internet Service Providers.
- 1 Gbps of commercial Internet access.
- TCP/IP performance tuning
- Expected throughput
- File transfer tools
- Network performance measurement tools
To learn more about transferring large data sets over the Internet, please visit the ESnet Network Performance Knowledge Base.