Robust multimodal pedestrian understanding algorithm for pedestrian safety in autonomous driving

The automotive industry is undergoing a revolutionary transformation with the rapid development of autonomous driving technologies. According to the European Commission's Strategy on Sustainable and Smart Mobility, more than 50% of new vehicles sold in the EU by 2030 are expected to feature advanced driver-assistance systems (ADAS) or full autonomous capabilities. This project focuses on the critical need for robust pedestrian understanding systems in autonomous vehicles by developing an advanced multimodal perception framework that significantly improves detection accuracy, behavior prediction, and safety in complex urban environments. Specially for the following key technical challenges: optimizing the multi-sensor fusion process to reduce feature redundancy while enhancing representation learning; expanding perception systems to incorporate rich semantic scene understanding for better contextual awareness; and developing defense mechanisms against adversarial attacks that could compromise system reliability. The proposed solution integrates cutting-edge deep learning techniques, including transformer-based architectures for efficient feature extraction, multi-task learning paradigms for simultaneous detection and prediction tasks, and adversarial training methods to enhance system robustness. The project's interdisciplinary approach, combining computer vision, machine learning, and transportation safety expertise, positions it to make significant contributions to both academic research and industrial applications in the emerging field of intelligent transportation systems. The expected results will significantly enhance the safety and reliability of autonomous vehicles in pedestrian-rich environments, contributing to the broader development of safer autonomous driving technologies. These advancements will support ongoing EU efforts to promote autonomous driving and build public trust in these systems.