Sulis: An EPSRC platform for ensemble computing delivered by HPC Midlands+

Computer simulation and modelling is increasingly seen as the third pillar of modern science, alongside theory and experiment. Increasingly powerful research computing facilities are required for this activity. Traditionally, the case for these facilities has been made through a scientific need to model larger physical systems, or simulate with increased fidelity. Such calculations benefit from larger and more powerful computers by exploiting ever-larger numbers of computational processing units (cores) within a single calculation. Sulis will support alternative and complementary ways of exploiting parallelism, specifically high throughput computing. Here the focus is on calculations of modest size, i.e. comparable to those which could be executed on a typical high-end workstation PC in a few days, but replicated thousands of times each running with different inputs or model parameters to solve a single problem. Working through this "ensemble" of calculations could easily take decades on a single multi-core PC, or many months with university level facilities. Sulis will allow researchers to complete workflows such as this in less than a week and hence apply their expertise to a broader range of problems and be reactive to availability of new input data. There are many computational tasks which fit into this "ensemble computing" model. One pertinent example is uncertainty quantification (UQ). Rather than simulate a single and likely imperfect model, UQ approaches generate ensembles of possible models and simulates them all. This allows predictions to be made statistically. The most likely outcome of the simulated process can be inferred from the ensemble of outputs, along with a confidence level based on the variability over the outputs. The latter is essential if using simulation as a design or decision making tool. A similar concept may be familiar from weather forecasting - models do not make absolute predictions but instead predict a probability of rain based on the fraction of simulations in which this occurs. This approach is applicable to a range of problems in the physical sciences, such as predicting material properties, yield of chemical processes, the motion of bacteria, fusion plasma stability etc. Other ensemble computing workflows include optimisation problems. Here each of the simulations independently searches a subset of the inputs/parameters for a model, reducing the time taken to locate viable solutions. This is essential, for instance, in studying disordered materials, far closer to the real world than the ideal perfect crystals assumed when seeking only a single solution. Ensemble computing is also used to generate, sample or process large datasets, often for subsequent use as inputs to train modern machine learning algorithms. For this reason Sulis will include a high-capacity multi-petabyte data storage capacity, exploiting modern solid-state storage technologies to reduce bottlenecks arising from reading and writing of data. It will also include a large number of graphics processing units (GPUs) - accelerator devices themselves now ubiquitous for machine-learning applications. A focus on ensemble computing raises challenges to researchers and software engineers. With thousands of simulations, the probability that at least one will fail is substantial. Software must be resilient to this failure. Similarly, managing the input and output of so many calculations can overload traditional data storage subsystems, requiring users to work with database technology rarely encountered by researchers outside of computer science departments. Hence a key feature of the Sulis service will be Research Software Engineering (RSE) support to assist and train users in tackling these problems, future-proofing the competitiveness of UK researchers to the challenges of computing at ever larger scales.