Today Dell EMC announced it will build a supercomputer to power Swinburne University of Technology’s groundbreaking research into astrophysics and gravitational waves. Called OzStar, the new supercomputer will be used by the ARC Centre of Excellence for Gravitational Wave Discovery (OzGRav) to understand the extreme physics of black holes and warped space-time.
We will be looking for gravitational waves that help us learn more about supernovas, the formation of stars, intergalactic gases and more,” said Professor Bailes. “It’s exciting to think that we as OzGRav could make the next landmark discovery in gravitational wave astrophysics – and the Dell EMC supercomputer will allow us to capture, visualize and process the data to make those discoveries.”
The supercomputer will process incredibly large volumes of data coming from giant telescopes, searching for insights that could unlock answers about the Universe, reaffirming Swinburne’s position as a global leader in physics research.
The OzStar supercomputer will feature the new Dell EMC 14th Generation PowerEdge Servers, Dell EMC H-Series Networking Fabric and Dell EMC HPC Storage with Intel Lustre filesystem, delivering more than 1 Petaflop of performance capability – or 31 million years of calculations in a single second.
Building on Einstein’s theory, 100 years on
Albert Einstein first predicted gravitational waves in 1915 in his theory of General Relativity. Based on theoretical physics alone, Einstein described how gravity warps and distorts space-time. It took another 100 years and huge advances in technology before gravitational waves were physically detected in 2015.
In this landmark discovery, scientists detected the ripples in space time caused by gravitational waves; these waves resulted from the collision of two black holes over a billion light years away. This confirmed Einstein’s theory in a very direct way – something even he doubted would ever be physically possible. It also demonstrated that time and space can be stretched and compressed by the passing of gravitational waves, which inspired excited discussion about other predictions of General Relativity, such as the nature of space and time in the vicinity of (and within) black holes.
While Einstein predicted the existence of gravitational waves, it took one hundred years for technology to advance to the point they could be detected,” said Professor Matthew Bailes, director of OzGRav, Swinburne University of Technology. “Discoveries this significant don’t occur every day and we have now opened a new window on the Universe. This machine will be a tremendous boost to our brand-new field of science and will be used by astrophysicists at our partner nodes as well as internationally.”
The AU$31.1 million ARC Centre of Excellence for Gravitational Wave Discovery was announced in September 2016 with the mission to capitalize on the historic first detections of gravitational waves. Led by Swinburne, it will use state-of-the-art technology to research and understand the extreme physics of black holes and warped space-time.
Powering a new field of gravitational wave astronomy
Swinburne selected Dell EMC technology for the AU$4 million supercomputer at the heart of this research. The system will be used to sift through the reams of data from Advanced LIGO, the observatory that first detected gravitational waves, to search for new examples of merger, explosions and even evidence of 1 cm “mountains” on nearby neutron stars that cause deviations in space-time.
The supercomputer will also serve our other astronomers both at Swinburne and nationally who calculate theoretical models and crunch data,” said Professor Jared Hurley, Centre for Astrophysics and Supercomputing, Swinburne University of Technology.
Up to 35% of the supercomputer’s time will be spent on OzGrav research related to gravitational waves. The supercomputer will also continue to incorporate the GPU Supercomputer for Theoretical Astrophysics Research (gSTAR), operating as a national facility for the astronomy community funded under the federal National Collaborative Research Infrastructure Scheme (NCRIS) in cooperation with Astronomy Australia Limited (AAL). In addition, the supercomputer will underpin the research goals of Swinburne staff and students across multiple disciplines, including molecular dynamics, nanophotonics, advanced chemistry and atomic optics.
Supercomputing power with scale and flexibility
Dell EMC has designed the supercomputer using its 14th Generation PowerEdge R740 server technology that scales from traditional enterprise deployment through to high performance computing and machine learning. The platform is used as a building block to both the management and computing nodes, while the server is based on the next-generation Intel Xeon scalable processors.
OzStar replaces the “green” machines that have served Swinburne for the last decade and seeks to further reduce Swinburne’s carbon footprint by minimising CO2 emissions by carefully considering heating, cooling and a very high performance per watt ration of power consumption. The modular and scalable design means the system can be simply managed to meet workload demands.
This combination of Dell EMC technologies will deliver the incredibly high computing power required to move and analyse data sets that are literally astronomical in size,” said Andrew Underwood, Dell EMC’s ANZ high performance computing lead, who collaborated with Swinburne on the supercomputer design. “Our 14th Generation PowerEdge server and H-Series networking will form the bedrock of this modern supercomputer, enabling Swinburne University of Technology to push the limits of compute and data-intensive research workloads on a single platform and drive faster research breakthroughs.”
The OzStar supercomputer will be built using three core building blocks:
- Dell EMC PowerEdge R740 compute and data-crunching nodes
- Containing Intel Xeon V5 processors and NVIDIA Tesla P100s
- Featuring the unique Dell EMC HPC BIOS setting to increase performance of the compute node
- Dell EMC H-Series Networking Fabric
- Designed to meet Swinburne’s performance, throughput and scalability requirements
- Powered by Intel’s Omni-Path Architecture (OPA) to provide 86.4 Terebits per second of aggregate network bandwidth at .9 µs (microsecond) latency
- Dell EMC HPC Storage with Intel Lustre filesystem
- Built on the tried and tested PowerEdge and PowerVault Modular Disk reference architecture for optimised capacity, throughput and reliability
Along with these ingredients, Dell EMC will deliver ProSupport engineers to provide 24/7 expert level support on each of the building blocks in the OzStar supercomputer.
Together, these building blocks will deliver Swinburne University of Technology with:
- 4,410 x86 cores at 2.3Ghz across 107 standard compute and eight data crunching nodes
- 230 x NVIDIA Tesla P100 12GB GPUs (one per CPU socket)
- 272 Intel Xeon Phi cores at 1.6Ghz across 4 C6320p KNL nodes
- A high-speed, low latency network fabric able to move data across each building block at over 100Gbps with various features to ensure reliability and traffic flow (9 Dell EMC H1048 switches), as well as GPU-Direct capability
- 5PiB of usable storage via the Lustre file system at 60GB/s throughput, on the Dell EMC PowerVault MD3060e storage with R740 controllers (10 Dell EMC PowerEdge R740 + 16 MD3060e and 1 MD3240)
Driving a new era of research
After confirming the project in mid-August, the Dell EMC supercomputer is expected to take four weeks to install and will be operational before the end of September.
Brilliant researchers are often only limited by advances in technology,” said Chris Kelly, vice president and general manager, Compute and Networking, Dell EMC Australia and New Zealand. “With this new supercomputer, Swinburne and OzGrav will be able to embark on a new era of astronomy that could unlock answers to questions mankind has pondered for centuries. It’s an incredibly exciting time for astronomical research and we at Dell EMC are proud to power the discoveries of tomorrow.”
