Immersion cooling and heat storage

High Performance Computers use substantial quantities of energy to keep them cool and operating efficiently. Buildings in the UK require substantial quantities of energy to keep their occupants warm. Today, energy for heating and cooling is carbon intensive, and nationally the supply of heat and cooling is responsible for 1/3 of the UK's greenhouse gases. Heat from cooling a HPC system can be used for space heating elsewhere, and heat storage can store excess heat, retrieving it when required. This proposal seeks to do just that: efficient cooling of HPC systems, and investigating storing the resulting excess heat in flooded coal mines - legacy assets from the UK's mining past acting as carbon-zero heat stores for a zero-carbon future. Here we examine the sustainable energy potential of combining complimentary energy demands - examining societal energy needs without costing the Earth. Current UK large HPC systems all use direct liquid cooling (DLC), where a cooling fluid is piped onto heat sinks within compute nodes, heat extracted and then exchanged via a heat exchanger before being released into the atmosphere. These systems can consume multiple Megawatts of power. Previous air-cooled systems were even less efficient. Immersion cooling is the natural progression in technology which involves fully immersing specially adapted servers in a heat transfer liquid, typically a mineral oil, which removes heat from all components (not just CPUs), simplifying server design (no fans, heat pipes and reduced embodied CO2. Heat is then extracted from the system at higher temperature. This technology uses as little as 5% of power for dealing with waste heat, much better than the ~20% for DLC systems and 40-100% for air-cooled systems, yielding a significant power saving. Current UK HPC data centres do not have any direct current experience with this relatively new technology, and therefore it is seen as a significant risk for adoption in new HPC systems. We therefore propose installing a commercial immersion cooling tank in Durham University's the data centre (which also hosts the EPSRC BEDE N8 Tier-2 system). This immersion test bed system will become a national testing facility for other data centre staff to visit to gain experience with this technology, and for vendors to demonstrate their server technologies. We will analyse the performance of kit hosted within this tank, and investigate properties such as fluid temperature and server energy consumption. A further benefit of immersion cooling is that the waste heat is at a higher (more usable) temperature than from conventional systems. Durham is the ideal location for such facility, currently hosting national ExCALIBUR hardware and enabling software systems and both Tier-1 and Tier-2 facilities. We furthermore propose to study data centre waste heat reuse, by investigating storing HPC waste heat in the abandoned and flooded coal mine workings beneath the data centre site. By storing the waste heat, it can be reused when required. This requires drilling several boreholes into the mine workings to abstract the water, add/subtract heat to it, and re-inject it back to the mine. In short, we would be investigating using the mine workings as a heat battery. This proposal is timely, since it fits well with the university's current plan to install a heat network across its campus. The site would be used as a living lab and exemplar for other potential systems.