DiaGrid is a large, multicampus distributed research computing network utilizing the HTCondor system and centered at Purdue University in West Lafayette, Indiana. In 2012, it included nearly 43,000 processors representing 301 teraflops of computing power. DiaGrid received a Campus Technology Innovators Award from Campus Technology magazine and an IDG InfoWorld 100 Award in 2009 and was employed at the SC09 supercomputing conference in Portland, Ore., to capture nearly 150 days of compute time for science jobs.
Contents
Partners
DiaGrid is a partnership with Purdue, Indiana University, Indiana State University, the University of Notre Dame, the University of Louisville, the University of Nebraska, the University of Wisconsin, Purdue's Calumet and North Central campuses, and Indiana University-Purdue University Fort Wayne. It is designed to accommodate computers at other campuses as new members join. The Purdue portion of the pool, named BoilerGrid, is the largest academic system of its kind.
Management
DiaGrid is managed by Information Technology at Purdue (ITaP), the central information technology organization at Purdue's West Lafayette campus, and ITaP's research computing unit the Rosen Center for Advanced Computing, which also operates the Steele, Coates, Rossmann, Hansen and Carter cluster supercomputers.
HTCondor
Through HTCondor, developed at the University of Wisconsin, DiaGrid harvests and manages computing cycles from idle or underused high-performance computing cluster nodes, servers, machines in campus computer and other labs, and office computers. Whenever a local user or scheduled job needs a given machine, the HTCondor job is stopped and automatically sent to another HTCondor node as soon as possible. While this "opportunistic" model limits the ability to do parallel processing and communications, a HTCondor pool can provide smaller, serial jobs vast numbers of cycles in a very short amount of time. HTCondor—and by extension, DiaGrid—is designed for high-throughput computing and is excellent for parameter sweeps, Monte Carlo simulation, or nearly any serial application. Some classes of parallel jobs (master-worker) may be run effectively via HTCondor as well.
Networking
To pool computational resources spread around Indiana and the Midwest, DiaGrid takes advantage of I-Light, the high-speed fiber-optic state network connecting Indiana campuses to each other, the Internet and national research networks such as the Internet2 and National LambdaRail. DiaGrid provides computational resources to researchers on both the Open Science Grid and the U.S. National Science Foundation's XSEDE system (formerly TeraGrid).
Uses
DiaGrid and BoilerGrid have been used by researchers at Purdue and elsewhere for a variety of purposes, such as imaging the structure of viruses at near-atomic resolutions, simulating the early stages of the Solar System's formation, projecting the reliability of Indiana's electrical supply, modeling the spread of water pollutants, discerning the structure of protein molecules and identifying millions of potential new forms of zeolites, silicate minerals widely used to catalyze chemical reactions on an industrial scale. DiaGrid also is being used to develop data processing techniques for the Large Synoptic Survey Telescope. Purdue added a Web-based portal for BLAST processing with DiaGrid in 2011.