DOME is a Dutch government-funded project between IBM and ASTRON in form of a public-private-partnership to develop technology roadmaps targeting the Square Kilometer Array (SKA), the world's largest planned radio telescope. It will be built in Australia and South Africa during the late 2010s and early 2020s. One of the 7 DOME projects is Microservers that are small, inexpensive and computationally efficient.
Contents
The goal of the microservers is the ability to be used both near the SKA antennas to do early processing of the data, and inside much larger supercomputers that will do the big data analysis. These servers can be deployed in very large numbers and in environmentally extreme locations such as in deserts where the antennas will be located and not in only in cooled datacenters.
A common misconception is that microservers offer only low performance. This is caused by the first microservers being based on Atoms or early 32bit ARM cores. The aim of the DOME microserver project is to deliver high performance at low cost and low power. The key characteristic of a microserver is the packaging: very small form factor, allowing short communication distances, eliminating all unnecessary components by integrating as much as possible from the compute server into a single SoC (Server on a chip). A microserver will not deliver the highest possible performance, instead, it offers an energy optimized design point at medium-high performance. Currently in 2015, several high performance SoCs are appearing on the market.
Design
In 2012 a team at IBM Research Zürich led by Ronald P. Luijten started pursuing a very computational dense, and energy efficient 64-bit computer design based on commodity components, running Linux. A system-on-chip (SoC) design where most necessary components would fit on a single chip would fit these goals best, and a definition of "microserver" emerged where essentially a complete motherboard (except RAM, boot flash and power conversion circuits) would fit on chip. ARM, x86 and Power Architecture based solutions were investigated and a solution based on Freescale's Power Architecture based dual core P5020 / quad core P5040 processor came out on top at the time of decision in 2012.
The concept is similar to IBM's Blue Gene supercomputers but the DOME microserver is designed around off the shelf components and will run standard operating systems and protocols, to keep development and component costs down.
The complete microserver is based on the same form factor as standard FB-DIMM socket. The idea is to fit 128 of these compute cards within a 19" rack 2U drawer together with network switchboards for external storage and communication. Cooling will be provided via the Aquasar hot water cooling solution pioneered by the SuperMUC supercomputer in Germany.
The designs of the first prototype was released to the DOME user community in July 3, 2014. The P5040 SoC chip, 16 GB of DRAM and a few control chips (such as the PSoC 3 from Cypress used for monitoring, debugging and booting) comprise a complete compute node with the physical dimmensions of 133×55 mm. The card's pins are used for a SATA, five Gbit and two 10 Gbit Ethernet ports, one SD card interface, one USB 2 interface, and power. The compute card operates within a 35 W power envelope with headroom up to 70 W. The bill of materials is around $500 for the single prototype.
Future
Late 2013 a new SoC was chosen for the second prototype. Freescale's newer 12 core T4240 is significantly more powerful and operates within the same power envelope as the P5040. The new prototype micro server card will house 24 or 48 GB of RAM, and be powered as well as cooled from the copper heatsink. It is being built and validated for the larger scale deployment in the full 2U drawer in early 2015. 128 microserver cards will fit inside a 2U water cooled drawer chassis and will contain 1536 cores and 3 or 6 TB of RAM. The bill of materials is estimated to be $650 for the second microserver prototype and around $10.000 for the drawer, including the water cooling apparatus, for a total cost of under $100.000 for a fully loaded system.
A team in South Africa will further enhance the microserver board to be "desert proof", i.e. being able to function in the harsh conditions of the desert where the SKA antennas will be placed. This includes ability to mitigate high temperatures and unstable power supply.