skip to content


21st International Conference on Information Fusion - 10 - 13 July 2018

To see the timetable for Tutorials, please click here.

An Introduction to Track-to-Track Fusion and the Distributed Kalman Filter

The increasing trend towards  connected  sensors (”internet of things” and ”ubiquitous computing”) derive  a demand  for powerful distributed estimation methodologies. In tracking applications, the ”Distributed Kalman Filter” (DKF) provides an optimal solution under certain conditions.  The optimal solution in terms of the estimation  accuracy is also achieved by a centralized fusion algorithm which receives either all associated measurements or so-called tracklets. However,  this scheme needs the result of each update step for the optimal solution  whereas the DKF works at arbitrary  communication  rates since the calculation is completely  distributed.  Two more recent methodologies  are based  on the ”accumulated  state densities” (ASD) which augment the states from multiple time instants. In practical applications, tracklet fusion based on the equivalent measurement often achieves reliable results even if full communication is not available. The limitations  and robustness of the tracklet fusion will be discussed.

At first, the tutorial will explain the origin of the challenges in distributed tracking. Then, possible solutions to them are derived and illuminated.  In particular, algorithms will be provided for each presented solution.

The list of topics includes: Short introduction to target tracking, Tracklet Fusion, Exact Fusion with cross-covariances, Naive Fusion, Federated Fusion, Decentralized Fusion (Consensus Kalman  Filter), Distributed Kalman Filter (DKF), Debiasing for the DKF, Distributed ASD Fusion, Augmented State Tracklet Fusion.

Presenter: Felix Govaers 

Analytic Combinatorics for Multi-Object Tracking and Higher Level Fusion

This tutorial is designed to facilitate understanding of the classical theory of Analytic Combinatorics (AC) and how to apply it to problems in multi-object tracking and higher level data fusion.  AC is an economical technique for encoding combinatorial problems—without information loss—into the derivatives of a generating function (GF).  Exact Bayesian filters derived from the GF avoid the heavy accounting burden required by traditional enumeration methods.  Although AC is an established mathematical field, it is not widely known in either the academic engineering community or the practicing data fusion/tracking community.  This tutorial lays the groundwork for understanding the methods of AC, starting with the GF for the classical Bayes-Markov filter.  From this cornerstone, we derive many established filters (e.g., PDA, JPDA, JIPDA, PHD, CPHD, MultiBernoulli, MHT) with simplicity, economy, and insight. We also show how to use the saddle point method (method of stationary phase) to find low complexity approximations of probability distributions and summary statistics.

Presenters: Roy Streit and Murat Efe


Extended Object Tracking:  Theory and Applications

Autonomous  systems are  an active  area of re- search  and technological  development.  These systems  require intelligence  and decision making,  including  intelligent  sensing, data collection and processing, collision  avoidance  and control. Autonomous   systems, especially  autonomous  cars need to be able to detect, recognise,  classify and track objects of interest, including  their location  and size.  In  the light of autonomous systems this tutorial will focus on tracking of extended objects, i.e., object tracking  using modern  high resolution  sensors  that give multiple  detections per object. State of the art theory will be introduced, and relevant real world applications will be shown where different  object types, e.g., pedestrians, bicyclists, and cars, are  tracked using  different sensors  such  as  lidar, radar, and camera.

Presenters: Karl Granström  and Marcus Baum


Implementations of Labeled Random Finite Set Multi-Target Filters

The random finite set framework for multi-sensor multi-target tracking has attached considerable interest in recent years. It provides a unified perspective of multi-target tracking in a very intuitive manner by drawing direct parallels with the simpler problem of single-target tracking. This framework has led to the development of well-known multi-target filters such as the Probability Hypothesis Density (PHD), Cardinalized PHD (CPHD), Multi-Bernoulli filters and a recent advance, the Generalized Labeled Multi- Bernoulli (GLMB) filter, which can handle in the order of one million targets in the presence of high clutter and misdetections.

The tutorial will present the Generalized Labeled Multi-Bernoulli (GLMB) filter and demonstrate the landmark example with one million targets. Further, we will show how the GLMB filter and an approximation called LMB filters are implemented in Matlab. Matlab code for these filters will be provided to all participants. It is envisaged that participants will come away with sufficient know-how to implement and apply these algorithms in their work.

Participants should have working knowledge of random variable, probability density function, Gaussian distribution, and concepts such as state space models.

Presenters: Ba-Ngu Vo and Ba-Tuong Vo

Information Quality in Information Fusion and Decision Making

Designing fusion systems for decision support in complex dynamic situations requires fusion of a large amount of multimedia and multispectral information to produce estimates about objects and gain knowledge of the entire domain of interest. Data and information to be processed and made sense of includes but is not limited to data obtained from physical sensors (infrared imagers, radars, chemical, etc.), human intelligence reports, and information obtained from open sources (traditional such as newspapers, radio, TV as well as social media such as Twitter, Facebook, and Instagram).

The problem of building such fusion-based systems is complicated by the fact that data and information obtained from observations and reports as well as information produced by both human and automatic processes are of variable quality and may be unreliable, of low fidelity, insufficient resolution, contradictory, and/or redundant. It can come from a broken sensor or a sensor improperly used in the environmental context. A message obtained from a human sensor can contain a human error or is intentionally sent to skew the information. Furthermore, there is often no guarantee that evidence obtained from the sources is based on direct, independent observations. Sources may provide unverified reports obtained from other sources (e.g., replicating information in social networks), resulting in correlations and bias. In the more malicious setting, some sources may coordinate to provide similar information in order to reinforce their opinion in the system. The fusion methods used can be insufficient to achieve the required rigor.

The success of decision making in a complex fusion driven human-machine system depends on how well knowledge produced by fusion processes represents reality, which in turn depends on how adequate data are, how good and adequate is the fusion model used, and how accurate, appropriate or applicable prior and contextual knowledge is.

The tutorial will discuss major challenges and some possible approaches addressing the problem of representing and incorporating information quality into fusion processes. In particular it will present an ontology of quality of information and identify potential methods of representing and assessing the values of quality attributes and their combination.  It will also examine the relation between information quality and context, and suggest possible approaches to quality control compensating for insufficient information and model quality.

Presenter: Galina Rogova

Machine and Deep Learning for Data Fusion

In this tutorial, I will present some techniques for fusion and analytics to process big centralized warehouse data, inherently distributed data, and data residing on the cloud. The broad range of artificial intelligence and machine and deep learning techniques to be discussed will handle both structured transactional and sensor data as well as unstructured textual data such as human intelligence, emails, blogs, surveys, etc., and image data. Specifically, the tutorial will explore Deep  Fusion  to  solve  multi-sensor  big  data  fusion  problems  applying  deep  learning  and artificial intelligence technologies.

As a background, this tutorial is intended to provide an account of both the cutting-edge and the most commonly used approaches to high-level data fusion and predictive and text analytics. The demos to be presented are in the areas of distributed search and situation assessment, information extraction and classification, and sentiment analyses.

Some of the tutorial materials are based on the following two books by the speaker: 1) Subrata Das. (2008). “High-Level Data Fusion,” Artech  House, Norwell, MA; and 2)  Subrata Das. (2014). “Computational Business Analytics,” Chapman & Hall/CRC Press.

Tutorial Topics include the following: High-Level Fusion, Traditional Machine Learning Algorithms, Popular Deep Learning Algorithms (e.g. Convolutional & Recursive Neural Networks, Deep Belief Networks and Restricted Boltzmann Machine, Stacked Autoencoder), Descriptive and Predictive Analytics, Text Analytics, Decision Support and Prescriptive Analytics, Cloud Computing, Distributed Fusion, Hadoop and MapReduce, Natural Language Query, Big Data Query Processing, Graphical Probabilistic Models, Bayesian Belief Networks, Distributed Belief Propagation, Text Classification, Supervised and Unsupervised Classification, Information Extraction, Natural Language Processing, Demos in R and Python.

Presenter: Dr Subrata Das

Multisensor-Multitarget Tracker/Fusion Engine Development and Performance Evaluation for Realistic Scenarios

While numerous tracking and fusion algorithms are available in the literature, their implementation and application on real-world problems are still challenging. Since new algorithms continue to emerge, rapidly prototyping them, developing for production and evaluating them on real-world (or realistic) problems efficiently are also essential. In addition to reviewing state-of-the-art tracking algorithms, this tutorial will focus on a number of realistic multisensor-multitarget tracking problems, simulation of large-scale tracking scenarios, rapid prototyping, development of high performance real-time tracking/fusion software, and performance evaluation on realistic scenarios. A unified tracker framework that can handle a number of state-of-the-art algorithms like the Multiple Hypothesis Tracking (MHT) algorithm, Multiframe Assignment (MFA) tracker and the Joint (Integrated) Probabilistic Data Association (J(I)PDA) tracker is presented. Modules for preprocessing (e.g., coordinate transformations, clutter estimation, thresholding, registration), data association (e.g., 2-D assignment, multiframe assignment, k-best assignment), filtering (e.g., Kalman filter, Interacting Multiple Model (IMM) Estimator, Unscented Kalman filter) and postprocessing (e.g., prediction, classification) are discussed. Fusion software with different architectures is also presented. Integration of sensors like radar, ESA, angle-only, PCL and AIS/ADS-B is demonstrated. Side-by-side performance evaluation of multiple algorithms using more than 30 metrics on realistic large-scale tracking scenarios is presented. A hands-on approach with ISR360, which is an end-to-end real-time software suite for Intelligence, Surveillance and Reconnaissance, will be the cornerstone of this tutorial.

The topics will include Review of Bayesian state estimation, Multitarget tracking system architecture, Implementation of J(I)PDA/MHT/MFA trackers, Implementation of a multisensor fusion engine, Implementation of realistic simulators, Implementation of a track analytics engine, Performance evaluation of trackers (MOP/MOE), and Real-world examples.

Participants should have a basic knowledge of tracking and fusion concepts.

Presenter: Professor T Kirubarajan 

Multitarget multisensor tracking: from traditional to modern distributed approach

The tutorial aims to present an historical overview of multisensor multitarget tracking from the traditional “divide & conquer” approach to a more modern approach based on the theory of finite random sets/Poisson point processes, but also aims to present the most recent research achievements of the group related to distributed tracking over peer-to-peer sensor networks. The talk will initially describe the intertwined R&D activities, along several decades, between academia and industry in conceiving and implementing - on live surveillance systems - tracking algorithms for targets in civilian as well as defense and security applications. In this respect, we trace back from the alpha-beta adaptive filter to modern random set filters passing through the Kalman  algorithm  (in  its  many  embodiments),  Multiple  Model  filters,  Multiple  Hypothesis Tracking, Joint Probabilistic Data Association, Particle filters. Then,  the  presentation  will  focus  on  recent  research  achievements  on  distributed  multitarget tracking over a peer-to-peer network consisting of the radio interconnection of multiple, possibly low-cost, devices with sensing, communication and processing capabilities. In this respect, fundamental issues like distributed fusion, handling of different fields of view, distributed sensor registration, etc., and relative solutions will be thoroughly investigated. Through the talk, applications to land, naval and airborne sensors will be mentioned. Further, active as well as passive radar experiences are overviewed.

The description will have a balanced look to both theoretical and practical implementation issues including mitigation of real life system limitations.

Presenter: Giorgio Battistelli, Luigi Chisci and Alfonso Farina

Multi Source and Multi Modal Sensor Fusion Strategies and Implementations in the world of Autonomous Driving

This tutorial is going to provide insights on the following sections:

  • Sensor fusion  levels  and  architectures   for  autonomous vehicles
  • Different environment  perception data and representation
  • Objects, Grids and Raw Data oriented sensor fusion problems
  • Nitty-gritty details that plays  a vital role in real life sensor fusion applications
  • Infrastructure  based sensor fusion

This tutorial is focussed towards  the stringent requirements, foundations,  development  and testing of sensor fusion algorithms  meant for advanced driver assistance functions, self-driving  car applications in automotive vehicle systems and vehicular infrastructure oriented sensor fusion applications . The audience  would be provided with the presentation materials used in the tutorial.

The complex sensor  world of autonomous  vehicles  is discussed in detail and different  aspects of sensor fusion problem related to this area is taken  as one of the core subject of this tutorial. In addition a special discussion section on a sensor fusion system that is designed to work on the data obtained from envoirnment perception sensors placed in an infrastructure   such as a parking house, is presented.

The audience  can see  the different representations  of the surrounding  environment   as percepted  by the het- erogeneous environment  perception sensors e.g. different kinds of  radar (multi-mode radar, short range radar), stereo  camera  and lidar. The relevant  state estimation algorithms, sensor fusion frameworks and the evaluation procedures  with  reference  ground truth are presented in detail. The audience can get a  first ever glimpse  of the data set obtained  from a sensor configuration   that would be used in the future Mercedes Benz autonomous vehicles.

Section on urban automated driving application with sup- port of infrastructure sensing, distributed computing, and cellular radio, is introduced. After a brief overview over the overall system and the individual  components, the hy- brid fusion design of the overall environmental perception for an automated vehicle comprising both onboard sen- sors and distributed  environmental  models delivered  via cellular radio, is presented. Advantages and disadvantages of different fusion architectures  for automated  driving with support  from infrastructure  sensing are discussed and the influence of cellular radio and overall system latency on different approaches  is presented.  After  a short discussion of possible approaches to incorporate  a mixture of both geo-referenced and vehicle-fixed  sensor data into a fusion system and the effect of ego localization errors on estimation  uncertainty  and this section with an introduction into behavior-generation  for automated vehicles which are supported by environmental  models received  via cellular radio and the limits of behavior- generation without the support of distributed environment perception. The interesting  part of the tutorial is covered  on the different challenging  and important real world imple- mentation  problems  and practical  aspects such as fusion with  incomplete information, data association,   sensor communication latency, real world testing, real-life like simulation etc. related to fusion and target tracking in automotive  setting. Challenges in automated driving in highway  and urban setting are discussed in detail during every  section of this tutorial. Interesting  research and application  based discussion on centralized, decentralized and hybrid-distributed  sensor fusion designs in particular to autonomous driving is discussed in depth using the results obtained  using several  real world data sets  that contains  various static and dynamic targets  would be presented in this tutorial. Fusion and management of the different  extended target and static object representations from heterogeneous  information sources  with different resolution is presented with examples.

Presenters: Bharanidhar Duraisamy, Ting Yuan, Tilo Schwarz, Martin Fritzsche and Michael Gabb

Multitarget Tracking and Multisensor Information Fusion

Objectives: To provide to the participants the latest state-of-the art techniques to estimate the states of multiple targets with multisensor information fusion. Tools for algorithm selection, design and evaluation will be presented. These form the basis of automated decision systems for advanced surveillance and targeting. The various information processing configurations for fusion are described, including the recently solved track-to-track fusion from heterogeneous sensors.

Presenter:  Yaakov Bar-Shalom

Noise Covariance Matrices in State Space Models: Overview, Algorithms, and Comparison of Estimation Methods

Knowledge of a system model is a key prerequisite for many state estimation, signal processing, fault detection, and optimal control problems. The model is often designed to be consistent with random behaviour of the system quantities and properties of the measurements. While the deterministic part of the model often arises from mathematical modelling based on physical, chemical, or biological laws governing the behaviour of the system, the statistics of the stochastic part are often difficult to find by the modelling and have to be identified using the measured data. Incorrect description of the noise statistics may result in a significant worsening of estimation, signal processing, detection, or control quality or even in a failure of the underlying algorithms.

The tutorial introduces a more than six decades long history as well as recent advances and the state-of-the-art of the methods for estimation of the properties (or statistical description) of the stochastic part of the model with a special emphasis on the state-space model noise covariance matrices estimation. The tutorial covers all major groups of the noise statistics estimation methods, including the correlation methods, maximum likelihood methods, covariance matching methods, and the Bayesian methods. The methods are introduced in the unified framework highlighting their basic ideas, key properties, and assumptions. Algorithms of individual methods will be described and analysed to provide a basic understanding of their nature and similarities. Performance of the methods will also be compared using a numerical illustration.

The attendees will be provided with course notes and sample implementations of the selected methods.

Presenters: Ondrej Straka, Jindrich Dunik and Jindrich Havlik

Object tracking, Sensor Fusion and Situational Awareness for Assisted- and Self-Driving Vehicles: Problems, Solutions and Directions

The automotive industry has been undergoing a major revolution in the last few years. Rapid advances have been made in assisted- and self-driving vehicles. As a result, vehicles have become more efficient and more automated. A number of automotive as well as technology companies are in the process of developing smart cars that can drive themselves. While totally self-driving cars are still in their infancy, some features like self-parking, proximity detection and lane identification have already made it into production in high-end vehicles. In spite of these recent developments, significantly more research is needed in order to perfect these nascent technologies and to make them ready for mass production. This provides the motivation for this tutorial.

In this tutorial, we aim to discuss a number of problems related to assisted- and self-driving vehicles, potential solutions and directions for research & development. The issues discussed in this tutorial will span multitarget tracking, multisensor fusion and situational awareness within the context of smart cars. We will also present some of the algorithms that are available in the open literature as well as those we have developed recently. In addition, we will also discuss related computational issues and sensor technologies. Finally, we will present some results on real data.

Presenter: Professor T Kirubarajan

Overview of High-Level Information Fusion Theory, Models, and Representations

Over the past decade, the ISIF community has put together special sessions, panel discussions, and concept papers to capture the methodologies, directions, needs, and grand challenges of high-level information fusion (HLIF) in practical system designs. This tutorial brings together the contemporary  concepts,  models,  and  definitions  to  give  the  attendee  a summary of the state-of-the-art in HLIF. Analogies from low-level information fusion (LLIF) of object tracking and identification are extended to the HLIF concepts of situation/impact assessment and process/user refinement. HLIF theories (operational, functional, formal, cognitive) are mapped to representations (semantics, ontologies, axiomatics, and agents) with contemporary issues of modelling, testbeds, evaluation, and human-machine interfaces. Discussions with examples of search and rescue, cyber analysis, and battlefield awareness are presented. The attendee will gain an appreciation of HLIF through the topic organization from the perspectives of numerous authors, practitioners, and developers of information fusion systems. The tutorial is organized as per the recent text:

E. P. Blasch, E. Bosse, and D. A. Lambert, High-Level Information Fusion Management and Systems Design, Artech House, April 2012.

Presenter: Erik Blasch

Statistical Methods for Information Fusion System Design and Performance Evaluation

Information fusion system find their application in multiple domains from defense applications to self-driving cars, and autonomous systems. Irrespective of the application domain, the design of an information fusion system requires evaluation of a multitude of design variables which have a direct impact on the fusion system performance. These variables include various sensor types and attributes, multiple tracking and fusion algorithms (i.e., low-level information fusion (LLIF) considerations), and different situation assessment and resource management approaches (i.e., high-level information fusion (HLIF) considerations).  It is imperative for the fusion system designers to identify the significant design decisions in this large design space and subsequently quantify their impact on the end fusion performance. Traditionally, information fusion community has taken a partitioned approach of isolated design and evaluation of the various attributes of the fusion system, which assumes a lack of interactions between design decisions. However, in complex systems, such as the information fusion system, interactions between system design variables, continue to dominate the performance. 

In this tutorial, a domain-agnostic framework, based on Design of Experiments, is presented which provides holistic performance evaluation of an information fusion system. This framework leverages systems engineering principles for identifying design variables which are then investigated by statistical methods (e.g., analysis of variance) for establishing statistical significance and quantifying their impact on fusion system performance. This tutorial will discuss theoretical foundations for performing design and analysis of experiments, followed by a hands-on information fusion system application example which can be transferred to domain-specific implementation in the participant’s area of interest. A refresher on Monte-Carlo simulations and hypothesis testing will also be provided. At the conclusion of the tutorial, the participants will be able to formulate an experimental design for the fusion system performance evaluation, employ hypothesis testing for comparing uncertain data, perform analysis of variance (ANOVA) to establish statistical significance of design variables and interactions, and perform multiple comparison range tests to quantify the impact of design variables and obtain sensitivity analysis of interactions for fusion system performance evaluation.

Presenters: Dr Ali Raz and Dr Daniel DeLaurentis