Next-Generation Visuohaptic Media Representation, Coding and Interactive Rendering for Immersive and Inclusive XR

XR-Sense aims to provide highly innovative and impactful enabling technologies for the next generation of virtual worlds. Based on a coordinated effort by leading scientific and industrial partners, XR-Sense will advance European XR capabilities through a unique combination of practical integration and breakthrough innovation. Today’s XR systems are constrained by static, pre-computed or sparse representations, which severely limits immersion, realism, and acceptance, as they cannot adapt to the dynamics of arbitrary, time-varying environments. XR-Sense will overcome these limitations by introducing a new visuo-haptic medium, where visual and tactile information are seamlessly integrated and jointly processed, coded, and rendered in real time. This paradigm shift will enable a more natural and convincing multimodal immersion experience. Scientific emphasis will be placed on intelligent processing (iPRO) of dynamic visuo-haptic scenes, including dense spatiotemporal reconstruction and compressed-domain operations, and on optimized coding and interactive rendering (iCORE), enabling adaptive streaming of editable XR scenes. Complementing these, iPASI will embed perceptual, affective, and social sciences into the system, ensuring inclusiveness and ethical and legal compliance by design. Technologies will be validated through three high-impact demonstrators in automotive, healthcare, and creative industries, covering autonomous driving support, remote surgery, and collaborative broadcasting/creativity. All developments will actively contribute to, and leverage, emerging XR interoperability standards (JPEG, MPEG, W3C, IEEE), ensuring long-term exploitation and sustainability. Beyond delivering world-class research results, XR-Sense will establish a distributed European XR innovation network, laying the foundation for disruptive applications, new markets, and Europe’s global leadership in multimodal virtual worlds.