IDEAS Factory - Detecting Terrorist Activities: Making Sense

The key challenge that this proposal addresses is the analysis and visualization of multiple sources of multi-modal data that may be partial, unreliable and contradictory. In addressing this challenge we expect to create an interactive visualization-based decision support assistant which collects data, fuses it, analyses it and visualizes the results in a way which can be shared by analysts. Each aspect of the assistant poses significant new scientific challenges. We envisage a systems architecture which has four main components:Collection of data encompasses automated approaches to gist-ing multimedia content (extraction of the gist of the information -- a process of abstraction which might, for example, summarise a phone message by the caller phone number, the phone number of the recipient, its duration, key words and phrases which stand out as most significant, and potentially suspicious words and phrases) and data management and resource allocation issues. Fusion and inference involves the integration of different modalities of data of variable reliability, the estimation of missing data for use in scenario development, methods for the resolution of contradictions, and psychological studies of how analysts relate information in this setting. We will investigate whether work on GIS which has developed meta-languages such as UnCertML can be used in this process.Analysis involves further summarizing of the fused data; we will build on existing machine learning techniques but anticipate that the characteristics of the data (which include temporal and spatial information as well as the uncertainty aspects discussed above) will pose significant new challenges. A particular challenge which will require psychological input is the drawing of relevant connections that have arisen in the fused data. Visualization must be informed by the operational model(s) of the data analyst(s), risk assessment and by legal considerations. The key challenge is to find a flexible, interactive way of visualising the data that allows the analyst to query the data and focus attention in a natural way. A key aspect of the visualization system is that it forms both the input and output of the system allowing the complexity of the data and underlying system to be hidden beneath an intuitive interface.Visual analytics is the emerging science of making sense of large data sets, that through the use of interactive visualization and query, supports the analytic reasoning process. The interactive visualization interface will support sense-making, query formulation, and information search by showing in visual representations associations and relationships between large, mixed-format and loosely-coupled data sets, such as un-structured reports, news feeds, photos, and structured databases. Very importantly as well, there is also a need for an intermediate layer that may draw on a variety of computing technologies e.g. Latent Semantic Analysis, ontologies) to enable the extraction of semantically meaningful relationships between data sets. In addition, by enabling changes in viewing perspectives (e.g. rotation, re-ordering, re-collating) it will facilitate the chance discovery of un-anticipated associations and resources. One of the original features of this proposal is that the system developed will be based on an analysis of user requirements undertaken by psychologists in the team. Psychological research into the behaviour of analysts has been undertaken elsewhere, but we believe this is the first research project that directly integrates psychological findings from research with analysts, into the development of a decision support system.