Scientific Supercomputer Network XSEDE Accelerated by Internet2 Upgrade

Scientists and engineers using XSEDE’s (Extreme Science, Engineering, and Discovery Environment) advanced infrastructure – supercomputers, data storage resources, software, networking and other compute resources – can easily generate extreme volumes of data. But if they cannot access resources or share data at efficient rates, the value of the infrastructure diminishes. 

“Advanced networking is critical… to support the researchers and educators who are making innovative use of our resources,” says John Towns, XSEDE project director, noting that the US National Science Foundation’s XSEDE project provides over 8,000 users with access to 17 supercomputers, data storage and management tools, and networking resources.

The NSF’s network of supercomputing sites now has the bandwidth to keep pace with discovery and innovation in the big data era. “Eliminating XSEDEnet’s previous, single backbone, we’ve improved our overall bandwidth between sites – and, now with Internet2, we’re part of a national infrastructure,” says Victor Hazlewood, deputy director of operations for XSEDE and chief operating officer at the National Institute for Computational Sciences at the University of Tennessee at Knoxville, US.

The new Internet2 network topology eliminates backbone contention and enables scientists and researchers to achieve higher transfer rates – 10 times faster than previously possible. The 100-gigabits-per-second Internet2 national network grew out of funding from the National Telecommunications and Information Administration (NTIA) for the US Unified Community Anchor Network (US UCAN) program. More than $62.5 million in federal stimulus funding was awarded, through NTIA’s Broadband Technology Opportunities Program, to national research and education networking organizations including Internet2.

XSEDEnet’s previous network topology provided a single 10 gigabits per second (10G) backbone, which limited end users from experiencing transfer performance near the full speed of 10G of bandwidth, “… unless they were the only ones on the network, which would rarely happen,” explains Hazlewood. “You can’t feed twelve lanes of traffic into a one-lane highway and expect traffic to flow.”

In addition to increased backbone bandwidth, another major benefit is access to Internet2’s software-defined networking (SDN) ready infrastructure. “This will allow us to provide SDN as a service,” says Hazlewood. “The XSEDE team has already begun research and proposals for equipment delivery at various test sites.” 

The XSEDE team will potentially be able to manage and allocate percentages of bandwidth. For example, if 50% of XSEDE end users are using GridFTP and taking up 90% of total bandwidth, then the XSEDE-wide file system (XWFS) may be starving for network resources. SDN-capable architecture and tools like OpenFlow will let XSEDE network engineers manage and schedule bandwidth on demand – both for wide area network applications like XWFS, as well as for extreme data transfers between specific sites.

XSEDE network engineers will be researching provisioning virtual network services with the goal of providing these capabilities for future production services. For example, if an engineer wants to define a network service between XSEDE service providers, one on the east coast and one on the west, SDN tools like OpenFlow provide the capabilities.

Virtual networks can be set up at a particular time, with a set amount of bandwidth, and for a certain duration. The setup could be permanent or torn down when the desired duration is met, or engineers could set up the virtual service so that it is provisioned dynamically.

“Previously, the methods for doing dynamic provisioning weren't as easy or as streamlined as they are now with OpenFlow,” says Ron Milford, manager of the InCNTRE SDN Lab at Indiana University, US.“You can now provision a new circuit in sub-second times, as opposed to previously where it could take 30 seconds or longer.”

“We are now set up for future work with researchers and scientists who want to move extreme amounts of data,” Hazlewood says. “But it’s just a stepping stone, as we work on evaluating SDN, and getting all end users connected at 100 gigabits to the new Internet2 100gb backbone. -- by Amber Harmon, © i SGTW