It started as an effort to circumvent the lag time associated with using U.S. Department of Energy computers, according to Dr. Srindhi Varadarajan of Virginia Tech’s Terascale Computing Facility, speaking at the O’Reilly Mac OS X Conference.
The project was to build an academic supercomputer. Budget contraints caused him to envision a linked system of off-the-shelf 64-bit processors, bound together with an off-the-shelf backbone.
His first call was to Dell. Negotiations for 64-bit Intel Itanium 2 processors fell through. And IBM said that its PowerPC 970 wouldn’t be available until 2004. Quotes of $9 million to $11 million, well over budget, eliminated AMD and HP.
The unlikely project savior appeared in June, when Apple announced the G5. Three days later, Varadarajan was on the phone with Apple and the next day, on a plane to Cupertino.
When Apple representatives asked Varadarajan how long he’d been using Macs, the answer was “never.” He said, “I’m probably one of the few people who came to the platform by reading the kernel manual.”
Apple committed within 24 hours, and he ordered 1,100 dual processor PowerMac G5s from the Apple Store for about $5 million. They were delivered the first of September, and the facility made its first calculations on September 23.
Fastest academic supercomputer in the world
The Terascale Computing Facility is the fastest supercompter in academia and the third-fastest in the world; it is the only one of the top 500 to run on Macs. The project team goal is to continue optimization so that they can meet the 10-teraflop mark. It can solve equations with 500,000 variables; its current benchmark is 9.555 teraflops.
“There is a lot of interest in Macs because everyone sitting in this domain of scientific computing realizes that here you have a processor that can really do numericals,” according to Varadarajan. “It can do numericals better than anybody out there, and that’s what they care about.”
“Had Apple’s announcement occurred two months prior, we would not even have looked at any of the other options,” he said. That’s because the other options would have cost twice as much and would have delivered lower performance.
Varadarajan explained the performance of the machines. Each G5 processor has two, double-precision, floating-point units; each is capable of a fused, multiple-add operation per cycle, which means 2 flops per cycle. Therefore, 2GHz corresponds to 8 GFlops; each dual G5 can deliver a peak of 16 GFlops of double-precision performance. This performance exceeds a modern Cray node.
One-tenth the cost of traditional supercomputer
Virginia Tech College of Engineering dean Hassan Aref told TechNewsWorld that the project would have cost as much as 10 times more ($50 million) had it used a traditional machine in the same class. The cost savings means that it is highly likely this setup will be emulated.
The supercomputer architecture used here mirrors a trend in IT toward distributed computing.
To cool the system, the project team in effect created a distributed refrigerator. Chillers cool water to about 40-50 degrees; this is then used to chill refrigerant, which is piped into a matrix of copper pipes.
The computers run with few customizations. About 100 volunteers were used to connect the machines; the machines are currently running stock Mac OS X 10.2.7. Varadarajan said the team does plan to upgrade to Panther (OS X 10.3).
Links: MacWorld (MacCentral); eWeek; TechNewsWorld; O’Reilly’s MacDev Center.
