Universities Create 'Tera-Scale' Supercomputer Grid

WEST LAFAYETTE, Ind., June 18 -- Purdue University and Indiana University (IU) have linked their IBM supercomputers in a computational grid via the universities' high-speed optical network, creating a facility capable of performing a trillion operations per second.

The process works by breaking complex programs into small segments, which are then "distributed" across hundreds of separate processors contained within the two supercomputers. As a result, the two combined computers are able to process ultracomplex applications that ordinarily would have been too large for either computer to handle separately.

When fully functional, the supercomputer network -- referred to as the Indiana Virtual Machine Room -- will be the first in the nation to tie together university-owned computers with a combined peak capacity of more than one teraflop, or more than a trillion operations per second, said James Bottum, Purdue's vice president for information technology.

The supercomputers are connected via the state's new high-performance, optical-fiber network, called I-Light, which enables the exchange of large amounts of information at the speed of light. Purdue and IU tested the system for the first time last month.

The supercomputer grid will enable researchers to perform innovative and massive new calculations, including the simulation of "synthetic environments," applications that help to predict how millions of people might react to situations ranging from product marketing to natural disasters. The tera-scale capability also will help scientists run complex simulations, such as those that model the behavior of materials at the atomic level or the effects of an earthquake in a metropolitan area, and it will enable the analysis of genomic data to help identify new treatments for human disease.

New Frontier

"What we've done here is a proof of concept for a system that pools computer resources," Bottum said. "We are pushing the computational frontiers, creating a computational grid that spans geographic boundaries."

IU's teraflop supercomputer and Purdue's IBM supercomputer contain more than 900 processors, for a combined peak theoretical capacity of more than 1.4 teraflops, said Michael McRobbie, vice president for information technology and CIO at IU.

"This computational grid will provide researchers at both institutions the benefit of larger total processing capacity than either university has alone," McRobbie said.

The linkage takes advantage of Purdue's large memory configuration and IU's raw computation power. Having the combined horsepower of two supercomputers will give the faculty and researchers at both institutions more flexibility in planning applications and sharing cycles and capacity.

"What we are effectively doing is building, virtually, a large computer out of multiple pieces," said David Moffett, Purdue's associate vice president for information technology for research computing services. "We are leveraging resources and using I-Light to eventually solve problems we couldn't have tackled before."

Applications that will be explored include environments called "synthetic economies," in which the behavior of millions of consumers can be predicted for a given economic scenario. The simulations, which are based on traditional military war-gaming, enable researchers and business people to see the consequences of their decisions and actions in real time. Possible applications include simulations that predict how consumers would respond to new promotional campaigns; changes in the pricing of particular products or the introduction of new products; what would happen if companies entered each others' markets; and how changes in technology, regulatory laws or consumer demand would affect particular markets.

The software that makes the complex simulations possible was developed by Alok R. Chaturvedi, an associate professor of management at Purdue's Krannert School of Management, and Shailendra Mehta, director of entrepreneurism and small business outreach at the Krannert School.

Synthetic Reality

"What we do in our synthetic environment is create artificial people," Chaturvedi said. "They are calibrated based on real data, and they behave just as people do in the real world.

"Now, what the distributed tera-scale environment will do is allow us to create artificial agents at very fine granularities. This advanced computing environment will enable us to create a synthetic environment that contains more elements, or more virtual people, and will provide a more accurate, detailed representation of the reality."

The tera-scale capability has enabled the researchers to expand the number of people in a synthetic environment into the millions, compared to hundreds for conventional applications. Tera-scale computation also allows synthetic environments to be changed on the fly to fit new applications, said Chaturvedi, who has been working on the software since 1993 and has used it to solve problems for the US Naval Air Command, US Army Recruiting Command and companies in the personal computer and agribusiness industries.

"Life sciences computing also presents extremely large and complex computational challenges," said Craig Stewart, director of research and academic computing at IU. "The linkage of the state's two largest university-owned supercomputers will make possible analyses by life scientists affiliated with the Indiana Genomics Initiative that would otherwise be impossible."

The supercomputer grid has been tested with fastDNAml, a program that infers evolutionary relationships from DNA sequence data. IU has previously distributed this program among Indiana, Singapore and Australia -- but on a limited-term basis. The universities' computational grid holds the potential of being a valuable computational resource that will enhance research in many scientific disciplines at both campuses and will help build the state's reputation as a hub of advanced information technology development.

Purdue has recently upgraded its IBM supercomputer through the IBM Shared University Research Program, which promotes research and strengthens ties between IBM and universities. Indiana University upgraded its IBM supercomputer to just more than 1 teraflop last year, making it the largest university-owned supercomputer in the US. This upgrade was made possible in part by a grant from IBM and funding made available for the Indiana Genomics Initiative by the Lilly Endowment Inc.