High order mathematical and computational infrastructure for streamed data that enhance contemporary generative and large language models

Context The DataSig II programme is an initiative aimed at enhancing the mathematical and computational frameworks for managing streamed data. At its core is Rough Path Theory (RPT), a mathematical field that has significantly impacted the understanding of complex, sequential data across various sectors. In recent years, models like Transformers have spectacularly transformed stream processing by handling long-range dependencies and creating parallelism, but transformers have notable limitations including high computational costs and inefficiencies in certain types of data streams. The programme brings together a team of researchers with a strong foundation in RPT, with an ambitious plan to further integrate this theory with modern machine learning. Introducing Contextual Controlled Differential Equations (CCDEs) to better model complex operations, such as translation, for continuous, multimodal streamed data. The Challenge the Project Addresses As the world becomes increasingly reliant on streamed data—ranging from financial transactions to cybersecurity logs—the challenge is to build models that can efficiently process, interpret, and act on this data in real-time. Traditional models, like transformers, while powerful, are often inefficient when dealing with very long sequences or highly complex data. They require large amounts of computational resources, and their lack of interpretability can pose risks in high-stakes environments like finance and cybersecurity. Detecting anomalous streams (such as fake speech) is another key challenge. Current methods struggle with the complexity of evolving, high-dimensional data streams, often resulting in false alarms when trying to identify unusual patterns. The project tackles these issues by developing mathematically grounded methods that are computationally efficient through the use of RoughPy. These methods offer better scalability, data representation, and anomaly detection capabilities. Aims and Objectives The primary aim of the DataSig II programme is to initiate a paradigm shift in the way streamed data is modelled and processed. The project focuses on creating innovative stream-to-stream models, like CCDEs, that integrate the strengths of RPT with modern machine learning techniques. The objectives are structured around four key themes: Abstracting Transformer Models: By developing new types of transformers that can handle rough, continuous streams of data, the project aims to improve the efficiency and adaptability of these models. Enhancing Data Representation: The project seeks to create more efficient methods for representing data streams, allowing for dimensionality reduction that makes models faster and more accurate, particularly when dealing with complex datasets. Detecting Anomalies: The team will build scalable methods for detecting outliers in data streams. These techniques will be designed to work across various industries and will be invariant to changes in data representation (such as unit transformations). Scaling Computation: The programme will also focus on scaling up existing computational tools, such as RoughPy, to support large-scale data processing on GPUs and FPGAs, improving real-time data handling capabilities. Potential Applications and Benefits The applications of this research span several key industries. Synthetic data generation will lead to better risk management and, improving the accuracy of trading algorithms in finance, enhanced tools for detecting anomalies in real-time will benefit Cybersecurity and can also be applied to critical tasks like malware detection and authenticating of digital content.