Research on Extended And Layered Multimodalities

REALM is a 36-month RIA that will define, develop, and demonstrate core technologies to establish a cross-media continuum towards the next generation of Virtual Worlds. Its vision is to deliver content that remains truly immersive, across diverse platforms and modalities, ultimately creating a profound and consistent sense of presence, regardless of the device. To achieve this, REALM adopts a human-centric methodology structured around three pillars: (1) Novel Capture Technologies, acquiring holistic environment data extending to scene geometry, material textures, temperature, surfaces, etc.; (2) Transformative Authoring Tools, leveraging this enhanced data for realistic, scene-based, intuitive interactions; and (3) Human-Centric Delivery Platforms, providing low-friction access across devices. Grounded in open standards and leveraging advances in Generative AI, volumetric reconstruction, real-time and haptic rendering, and networked systems, these interoperable tools will build a shared framework to enable media convergence and a novel concept of Virtual Worlds.REALM brings together Europe’s leading institutions in optics & photonics, 3D graphics, networking, HCI, haptics, and immersive arts—covering the entire value chain. The project will generate high-value impact for Europe’s creative industries, with spillover into industrial training, advanced manufacturing, and healthcare. Ultimately, REALM will contribute to a sovereign, interoperable, and human-centric European ecosystem for Virtual Worlds.