Operators Cray Inc. | Operating system | |
 | ||
Architecture AMD Opteron 6200 Interlagos series 16-core CPUs
Nvidia Tesla K20 Kepler series GPUs Power 45 to 54.1 kW per cabinet Memory 16 or 32 GB ECC DDR3 per CPU
5 or 6 GB ECC GDDR5 per GPU Speed Theoretical peak of 50 petaFLOPS | ||
XK7 is a supercomputing platform, produced by Cray, launched on October 29, 2012. XK7 is the second platform from Cray to use a combination of central processing units ("CPUs") and graphical processing units ("GPUs") for computing; the hybrid architecture requires a different approach to programming to that of CPU-only supercomputers. Laboratories that host XK7 machines host workshops to train researchers in the new programming languages needed for XK7 machines. The platform is used in Titan, the world's second fastest supercomputer in the November 2013 list as ranked by the TOP500 organization. Other customers include the Swiss National Supercomputing Centre which has a 272 node machine and Blue Waters has a machine that has Cray XE6 and XK7 nodes that performs at approximately 1 petaFLOPS (1015 floating-point operations per second).
Contents
Overview
XK7 is scalable up to 500 cabinets, each contains 24 blades and each blade contains 4 nodes (1 CPU and 1 GPU per node). The CPUs available are of the 16-core AMD Opteron 6200 Interlagos series and the GPUs are of the Nvidia Tesla K20 Kepler series. Each CPU can be paired with either 16 or 32 GB of error-correcting code memory (ECC) while the GPUs have either 5 or 6 GB of ECC memory depending on the model of GPU used. The nodes communicate with each other via the Gemini Interconnect; each Gemini chip services 2 nodes with a capacity of 160 GB/s. Depending on the components used, a full cabinet will consume between 45 and 54.1 kW of electricity which is converted into heat; thus the cabinets need cooling, either by air or water.
XK7 based machines run the Cray Linux Environment which incorporates SUSE Linux Enterprise Server. Code to run on an XK7 machine can be written in a range of programming languages. The hybrid architecture requires different programming to conventional CPU-only supercomputers; Oak Ridge National Laboratory and the Swiss National Supercomputing Centre hold workshops to educate researchers on the new programming approach.
Usage
The XK7 platform was announced on October 29, 2012 to coincide with the completion of Titan at Oak Ridge National Laboratory (ORNL). Titan has 18,688 XK7 nodes, each containing an Opteron 6274 CPU with 32 GB of memory and a K20X GPU with 6 GB. The computer has a theoretical peak performance of 27.1 petaFLOPS but in the LINPACK benchmark used by the TOP500 organisation to rank supercomputers it performed at 17.59 petaFLOPS, enough to take first place on the November 2012 list. Titan uses 8.2 MW of electricity and is third on the Green500 list which ranks supercomputers by their energy efficiency.
The National Center for Supercomputing Applications (NCSA) in Illinois has a machine, Blue Waters, using a combination of Cray XE6 and XK7 nodes. The machine has 3072 XK7 nodes and 22,752 XE6 nodes. Each XE6 node has two Opteron 6276 and 32 GB of memory per CPU. The XK7 nodes also have Opteron 6276 CPUs with 32 GB of memory and a K20X GPU with 6 GB. Blue Waters has performed at over 1 petaFLOPS in benchmarks however the project managers do not believe in the relevance of the LINPACK benchmark used by the TOP500 organisation therefore did not submit a benchmark test for ranking.
The Swiss National Supercomputing Centre (CSCS) machine named Todi was upgraded to XK7 on October 22, 2012. Todi has 272 nodes with Opteron 6272 CPUs with 32 GB of memory and a K20X GPU with 6 GB. Todi has a theoretical peak performance of 393 teraFLOPS and performed at 274 teraFLOPS in the November 2012 TOP500 list taking 91st place. Todi consumes 122 kW and is ranked fourth, one behind Titan, on the November 2012 Green500 list.