Robust remote sensing for multi-modal characterisation in nuclear and other extreme environments

This project addresses the problem of "characterisation" of Extreme Environments (EE), by deploying and combining information from a variety of different Remote Sensing modalities. Our principle application area is nuclear decommissioning, however our research outputs will be relevant to other EE. Before nuclear decommissioning interventions can happen, the facility/plant being decommissioned must be "characterised", to understand: physical layout and 3D geometry; structural integrity; contents including particular objects of interest (e.g. fuel rod debris). 3D plant models must further be annotated with additional sensed data: thermal information; types/levels/locations of contamination (radiological, chemical etc.). Characterisation may be needed before, during or after POCO (Post Operation Clean Out). "Quiescent buildings" may be over half a century old, with uncertain internal layout and contents. Characterisation is needed in dry environments (e.g. contaminated concrete "caves") and wet environments (e.g. legacy storage ponds). Caves may be unlit, causing difficult vision problems (shadows, contrast, saturation) with robot-mounted spotlights. Underwater environments cause significant visibility degradation for RGB cameras, and render most depth/range sensors unusable. New technologies, e.g. acoustic cameras, engender interesting new challenges in developing algorithms to process these new kinds of image data. In many cases, robots are needed to deploy Remote Sensors into Extreme Environments and move them to desired locations and viewing poses. In some cases, robots must also assist characterisation by retrieving samples of contaminated materials. In many case real-time Remote Sensing data must also be applied to inform and control the actions of robots, while performing remote intervention tasks in EE. This project brings together a unique, cross-disciplinary and international team of researchers and institutes, spanning three continents, to address these challenges. End-users NNL and JAEA will advise on scenarios and challenges for Remote Sensing in nuclear environments. Active facilities at JPL will be used to measure degradation of sensors, chips and software under a variety of radiation types and doses. JPL and Essex researchers will use this data to develop new models for predicting such degradation. Essex researchers will then develop new methods for software and embedded hardware design, which overcome radiation damage by incorporating new approaches to fault detection, tolerance and recovery. The scenarios provided by the partners, and the degradation data measured by JPL, will be used to develop new benchmark data-sets comprising data from multiple sensing modalities (RGB cameras, depth/range cameras, IR thermal imaging, underwater acoustic imaging), featuring a vairiety of nuclear scenes and objects. UoB and Essex researchers will develop new algorithms for real-time 3D characterisation of scenes, with intelligent and adaptive fusion of multiple sensing modalities. First, new multi-sensor fusion methods will be developed for 3D modelling, semantic/meta-data labelling, recognition and understanding of scenes and objects. Second, these methods will be extended to incorporate new algorithms for overcoming extreme noise and other kinds of degradation in images and sensor data. Third, we will develop the robots and robot control methods needed to: i) deploy remote sensors into extreme environments; ii) exploit remote sensor data to guide robotic interventions and actions in these environments. Finally, we will carry out experimental deployments of these new technologies. Robust hardware and software solutions, developed by Essex, will be tested in active radiation environments at JPL. We will also carry out experimental robotic deployments of sensor payloads into inactive but plant-representative nuclear environments at NNL Workington and the Naraha Fukushima mock-up testing facilities in Japan.