This framework enables high-speed data transfer optimization based on real-time programmable data plane telemetry
March 23, 2021
Energy Science Network (ESnet) and Florida International University (FIU) received a two-year National Science Foundation (NSF) Office of Advanced Cyberinfrastructure research grant for “Q-Factor: A framework to enable high-speed data transfer optimization based on real-time network state information provided by programmable data planes.” (NSF Award Abstract #2018754)
As the trend towards data-intensive research continues, scientists and campus IT experts are investing significant resources to facilitate the efficient movement of large datasets between geographically distributed sites. Because many university campus networks are not currently configured to support the data movement requirements of data-intensive science, campus researchers require new tools to address poor network performance.
To help address this problem, FIU and ESnet are planning to research, prototype, and deploy new technology named “Q-Factor.” The central focus of Q-Factor research will be the creation of a novel telemetry agent that will be capable of dynamically tuning data transfer parameters at rapid sub-second intervals using temporal network state information collected from programmable data planes. Q-Factor will enhance inter-campus data transfers performed over regional, national, and international networks without requiring changes to end-user applications.
Science applications, such as the Vera Rubin Observatory, the Open Science Grid, the Large Hadron Collider (LHC) high luminosity experiments, and big data science drivers will benefit from the increased bandwidth utilization provided by Q-Factor.
“The cost of achieving effective end-to-end performance is high for network operators, because of the lack of real-time network state information. Q-Factor is an exciting research opportunity for FIU and ESnet to create knowledge that will benefit domain scientists and their network-dependent applications,” said Julio Ibarra, Q-Factor co-principal investigator and assistant vice president of technology augmented research at FIU.
“At ESnet, we are excited to partner with FIU to further advance our network’s capabilities, increase the performance of scientific data transfers, and unleash the potential in programmable data plane telemetry,” said Richard Cziva, Q-Factor co-principal investigator and software engineer at Lawrence Berkeley National Laboratory.
About ESnet: ESnet provides the high-bandwidth, reliable connections that link scientists at national laboratories, universities, and other research institutions, enabling them to collaborate on some of the world’s most important scientific challenges including energy, climate science, and the origins of the universe. Funded by the DOE Office of Science, ESnet is managed and operated by the Scientific Networking Division at Lawrence Berkeley National Laboratory. As a nationwide infrastructure and DOE User Facility, ESnet provides scientists with access to unique DOE research facilities and computing resources (https://www.es.net/).
About FIU: Florida International University is Miami’s public research university, focused on student success. According to U.S. News and World Report, FIU has 26 top-50 rankings in the nation among public universities and Washington Monthly Magazine ranks FIU among the top 20 public universities contributing to the public good and No. 12 for social mobility. FIU is a top U.S. research university (R1), with more than $200 million in annual expenditures. FIU ranks 15th in the nation among public universities for patent production, which drives innovation, and is one of the institutions that helps make Florida the top state for higher education. The Next Horizon fundraising campaign is furthering FIU’s commitment to providing students Worlds Ahead opportunities. Today, FIU has two campuses and multiple centers, and supports artistic and cultural engagement through its three museums: Patricia & Phillip Frost Art Museum, the Wolfsonian-FIU, and the Jewish Museum of Florida-FIU. FIU is a member of Conference USA, with more than 400 student-athletes participating in 18 sports. The university has awarded more than 330,000 degrees to many leaders in South Florida and beyond. For more information about FIU, visit www.fiu.edu
About CIARA: Florida International University’s Center for Internet Augmented Research and Assessment (CIARA), in the Division of IT, exists as a vehicle to contribute to the pace of research and the quality of education at FIU. CIARA is an evolving interconnection point between research scientists, graduate students, and faculty. These interconnections foster a domain-specific tailoring of Internet technologies to support faculty research and, in the process, improve graduate education. Institutions affiliated with CIARA use an interconnecting cyber infrastructure that ties together science and engineering research and education communities in the U.S., with peer communities in Latin America, the Caribbean, and Sub-Saharan Africa.
About AmLight ExP: AmLight Express and Protect (ExP) implements a hybrid network strategy that combines optical spectrum (Express) and leased capacity (Protect) that builds a reliable, leading-edge diverse network infrastructure for research and education. AmLight ExP operates high-performance network links connecting the USA with Latin America and Africa, funded by the National Science Foundation (NSF award OAC-2029283), with significant investments from the Research and Education Network at Sao Paulo (Rednesp) (FKA Academic Network of São Paulo), Brazil’s National Research and Education network (RNP), the Association of Universities for Research in Astronomy (AURA), the regional network of Latin America (RedCLARA), and national R&E network of South Africa (TENET/SANReN). The total bandwidth provided by AmLight ExP between the U.S. South America, and Africa is expected to grow to over 4 Tbps in aggregate capacity from 2020-2025. This flexible inter-regional infrastructure enables science communities to expand research and learning activities, empowered through access to scalable optical spectrum on submarine cables and programmable networks. (http://www.amlight.net/)