Navigating the Granite Pass: How Different Sensor Fusion Workflows Handle Data Obstacles

Sensor fusion combines data from multiple sensors to produce a more accurate and reliable estimate of the world. In practice, the world is messy: sensors drop out, data arrives late, and measurements sometimes contradict each other. For teams building autonomous systems, the choice of fusion workflow often determines whether the system gracefully degrades or catastrophically fails when obstacles arise. This guide compares three major sensor fusion workflows—Kalman filter-based, particle filter, and deep learning fusion—through the lens of a common challenge: navigating a rocky mountain pass with intermittent GPS, noisy lidar, and a drifting IMU. We show how each workflow handles data obstacles, where they struggle, and how to choose the right one for your project.

Why Data Obstacles Matter in Sensor Fusion Workflows

Modern autonomous systems rely on a suite of sensors: cameras, lidars, radars, IMUs, GPS receivers, and more. Each sensor has its own failure modes. GPS signals can be blocked by terrain or buildings. Lidar returns degrade in fog or heavy dust. IMU measurements drift over time. Cameras suffer from motion blur and low light. The job of a sensor fusion workflow is to combine these imperfect signals into a coherent state estimate. When one sensor fails, the workflow must decide how much to trust the remaining sensors, and how to handle the missing or conflicting information.

A poorly designed fusion workflow can amplify errors. For example, if a Kalman filter trusts a drifting IMU too much during a GPS outage, the position estimate will drift rapidly. On the other hand, if the filter overcorrects when GPS returns, it can introduce jumps that destabilize control. These are not theoretical problems: practitioners report that sensor dropout and temporal misalignment are among the top causes of field failures in autonomous vehicles and robotics.

Understanding how different fusion workflows handle these obstacles is critical for engineers and researchers designing robust perception systems. The choice is not just about accuracy on ideal datasets, but about graceful degradation under real-world conditions. This guide is for anyone who has to select, implement, or tune a sensor fusion pipeline for a mobile robot, drone, or autonomous vehicle.

The Three Workflows We'll Compare

We focus on three widely used approaches: the extended Kalman filter (EKF), the particle filter (also known as sequential Monte Carlo), and deep learning-based fusion (typically using recurrent neural networks or transformers). Each has distinct strengths and weaknesses when facing data obstacles.

Core Idea: How Fusion Workflows Handle Data Obstacles

At its heart, sensor fusion estimates a hidden state (position, velocity, orientation) from noisy measurements. Data obstacles come in several forms: missing data (a sensor stops sending updates), delayed data (a measurement arrives after the system has moved on), outlier data (a measurement far from expected), and conflicting data (two sensors report incompatible values). The core idea behind any fusion workflow is to model the uncertainty of each measurement and update the state estimate in a way that is robust to these imperfections.

Kalman filter-based workflows handle obstacles by maintaining a covariance matrix that represents uncertainty. When a sensor drops out, the filter propagates the state using the motion model, and the covariance grows, reflecting increased uncertainty. When a delayed measurement arrives, the filter can incorporate it using a retrodiction step (smoothing) or ignore it if the delay exceeds a threshold. Outlier rejection is done via Mahalanobis distance gating: if a measurement is too far from the predicted state, it is discarded. This approach works well when the system dynamics are well-modeled and the noise is Gaussian.

Particle filters use a set of weighted particles to represent the state distribution. They handle missing data naturally: when no measurement arrives, the particles propagate according to the motion model, and their weights remain unchanged. For delayed or out-of-order measurements, particle filters can be more flexible by resampling and reweighting when the data arrives. Outliers are handled by the likelihood function: a measurement that is far from all particles will assign very low weights, effectively ignoring it. However, particle filters require careful tuning of the number of particles and can suffer from degeneracy (all weight on one particle) in high-dimensional spaces.

Deep learning fusion workflows learn to map sequences of sensor data to state estimates directly. They can handle missing data by training with dropout or masking, and they can learn complex patterns of sensor failure. For example, a recurrent neural network can learn to predict the state even when lidar is absent for several time steps, by relying on IMU and camera data. Outliers are handled implicitly if the training data includes realistic anomalies. However, deep learning approaches are data-hungry, hard to interpret, and can fail unpredictably on out-of-distribution scenarios.

The Common Thread: Uncertainty Representation

All three workflows represent uncertainty, but in different forms. Kalman filters use Gaussian covariance, particle filters use a set of discrete samples, and deep learning methods often output a distribution (e.g., a mixture of Gaussians). The way uncertainty is represented directly affects how each workflow responds to data obstacles. For instance, a Kalman filter's Gaussian assumption can be limiting when the true distribution is multimodal (e.g., when the robot could be in two possible locations after a long GPS dropout). Particle filters handle multimodality naturally, while deep learning methods can learn multimodal outputs if trained on data that exhibits them.

How It Works Under the Hood

We now examine the mechanics of each workflow and see how they process sensor data when obstacles arise. Consider a simplified autonomous rover scenario: the rover traverses a mountain pass with a known terrain map. It has a GPS receiver (1 Hz, accurate to 5 m), a lidar (10 Hz, range 50 m, occasional dropouts in dust), and an IMU (100 Hz, gyro bias drift 0.1 deg/s). The goal is to estimate the rover's position and heading in real time.

Kalman Filter Workflow

An EKF fuses IMU data as a control input (predict step) and GPS/lidar as measurement updates. When GPS is available, the filter updates the position and reduces uncertainty. When GPS drops out (e.g., under a cliff), the filter relies on IMU dead reckoning. The covariance grows quadratically with time, reflecting the drift. If lidar also drops out due to dust, the filter has only IMU data. The filter can still run, but the uncertainty grows rapidly. When GPS returns, the filter incorporates it, and the state jumps back toward the true position. This jump can be abrupt, causing control issues. To mitigate, some implementations use a smoother or limit the innovation.

For delayed lidar measurements (e.g., due to processing lag), the EKF can use a buffer of past states and re-update them (a technique called delayed-state EKF). This is computationally expensive but can handle delays up to a few time steps. Outlier rejection is done by comparing the innovation (measurement minus predicted) to the innovation covariance. If the Mahalanobis distance exceeds a threshold (e.g., 3 sigma), the measurement is ignored.

Particle Filter Workflow

A particle filter represents the rover's state as 1000 particles. Each particle has a weight proportional to the likelihood of the measurements. When GPS is lost, the particles propagate using IMU data, and their weights remain equal. The spread of particles increases over time, representing growing uncertainty. When lidar is available, the weight of each particle is updated based on how well it matches the lidar scan (e.g., using a scan-matching likelihood). If lidar drops out, only IMU drives the particles, and the distribution widens.

When GPS returns, the weights are updated based on the GPS measurement. Particles far from the GPS reading get low weights, and resampling concentrates particles near the GPS position. This naturally handles the jump without abrupt state changes—the distribution shifts gradually through resampling. For delayed measurements, the particle filter can store past particles and reweight them when the measurement arrives, though this increases memory and computation. Outlier rejection is inherent: a spurious GPS reading will assign low likelihood to most particles, and those particles will be resampled away, but if the outlier is persistent, it can cause the filter to diverge. To guard against this, some particle filters use a robust likelihood function (e.g., a mixture of a Gaussian and a uniform distribution) to reduce the influence of outliers.

Deep Learning Fusion Workflow

A deep learning model, such as a gated recurrent unit (GRU) or a transformer, is trained on sequences of sensor readings and ground-truth poses. During inference, the model takes the latest window of sensor data (e.g., the last 50 time steps) and outputs the current pose. If GPS is missing, the model can still predict using lidar and IMU. If lidar is also missing, it relies solely on IMU, but the model may have learned that IMU-only predictions are less accurate. The model can handle missing data by using a mask indicator or by training with random sensor dropout.

Delayed measurements are problematic for deep learning models because they expect synchronous inputs. A common workaround is to buffer measurements and feed them asynchronously, but this increases the input dimension and complexity. Outlier rejection is not explicit; the model must learn to ignore outliers from training data. If the training data contains realistic outliers, the model may learn to downweight them. However, if the outlier pattern is different at test time (e.g., a new type of interference), the model can fail unpredictably.

Worked Example: Navigating the Granite Pass

We put these workflows to the test with a concrete scenario. Our rover traverses a narrow canyon (the Granite Pass) with steep walls that block GPS for 30 seconds. During this time, lidar occasionally drops out for 2–3 seconds due to dust. The IMU has a constant gyro bias of 0.1 deg/s and accelerometer noise. The ground truth path is a gentle curve.

Scenario Timeline

1. t=0 to t=10: GPS available, lidar clear. All workflows track well.
2. t=10: GPS lost (canyon walls). Lidar continues.
3. t=15 to t=18: Lidar dropout (dust cloud). Only IMU.
4. t=18: Lidar returns.
5. t=40: GPS returns (end of canyon).

Kalman Filter Performance

During the GPS loss (t=10 to t=40), the EKF relies on IMU and lidar. From t=10 to t=15, lidar updates keep the position accurate to about 1 m. At t=15, lidar drops out: the filter dead-reckons with IMU. The covariance grows quadratically. By t=18, the position error is about 5 m. When lidar returns, the filter corrects, but the jump is moderate (2 m). At t=40, GPS returns: the innovation is large (10 m), and the state jumps abruptly. This can cause control instability. The filter recovers, but the jump is unsettling.

Particle Filter Performance

The particle filter (1000 particles) behaves similarly during the GPS loss, but because particles spread out, the distribution captures the uncertainty. At t=15, the particles spread into a cloud around the estimated path. When lidar returns, the weights concentrate, but some particles remain in the wrong area, causing a slight lag in convergence. At t=40, when GPS returns, the particles far from the GPS reading get low weights, and resampling quickly moves the cloud to the correct location. The transition is smoother than the Kalman filter, with no abrupt jump, but the estimate may be slightly biased for a few seconds.

Deep Learning Performance

A GRU model trained on similar trajectories (but not exactly this canyon) tracks well initially. During the lidar dropout, the model relies on IMU and its learned dynamics. Because the model has seen dropout patterns in training, it maintains a reasonable estimate (error 3 m after 3 seconds). When lidar returns, the model integrates the new data smoothly, with no jump. At t=40, GPS returns: the model's prediction aligns well with the GPS reading, with only a small correction. However, if the model had not been trained on GPS dropout of this duration, it might over-rely on IMU and drift more. The deep learning workflow shows the best smoothness but requires extensive training data covering the exact failure modes.

Key Takeaways from the Example

The Kalman filter is simple and deterministic but suffers from abrupt jumps when sensors return. The particle filter handles the transition more gracefully but requires more computation and can have residual bias. The deep learning model provides the smoothest estimates but is brittle to untrained scenarios. In practice, many teams use a hybrid: a Kalman filter with robust outlier rejection and a smoother, or a particle filter with fewer particles and a proposal distribution from a neural network.

Edge Cases and Exceptions

No fusion workflow is perfect. Here are some edge cases that push each approach to its limits.

Multi-Modal Conflict

When two sensors give conflicting but equally plausible measurements (e.g., GPS says position A, lidar says position B, both with high confidence), a Kalman filter will average them, potentially producing a position that is inconsistent with both. A particle filter can maintain multiple modes (clusters of particles) if the likelihood function supports both. Deep learning models may produce a single prediction that averages the two, or a multimodal output if trained to do so. In practice, conflict resolution often requires a higher-level reasoner or a robust cost function.

Asynchronous Data Rates

Real systems have sensors with different rates and unpredictable latencies. Kalman filters handle this naturally by processing measurements as they arrive, using the time stamp to predict the state to the measurement time. Particle filters can do the same, but resampling at irregular intervals can cause particle depletion. Deep learning models expect fixed-rate inputs; handling asynchronous data requires interpolation or a recurrent architecture that can handle variable time steps (e.g., a continuous-time model).

Moving Sensor Platforms

If the sensors themselves move relative to the vehicle (e.g., a pan-tilt lidar), the fusion workflow must account for the sensor pose. Kalman filters can model this with an augmented state, but it increases complexity. Particle filters can incorporate sensor motion by transforming particles to the sensor frame. Deep learning models can learn the sensor motion if it is consistent, but sudden changes (e.g., a camera that moves) can confuse the model.

Non-Gaussian Noise

Kalman filters assume Gaussian noise. In practice, sensor noise can be heavy-tailed (e.g., lidar multipath reflections). Particle filters can handle any noise distribution by defining an appropriate likelihood. Deep learning models can learn the noise distribution from data, but they may overfit to the training noise characteristics.

Limits of Each Approach

Understanding the limits of each fusion workflow helps avoid costly mistakes.

Kalman Filter Limits

The Gaussian assumption is the biggest limit. In environments with non-Gaussian noise or multimodal distributions, the EKF can diverge. The linearization (first-order Taylor expansion) can also cause errors in highly nonlinear systems. Additionally, the EKF requires careful tuning of process and measurement noise covariance matrices, which is often done manually. It is also sensitive to initialization: a poor initial state can cause the filter to converge slowly or diverge.

Particle Filter Limits

The number of particles grows exponentially with the state dimension (curse of dimensionality). For high-dimensional state spaces (e.g., full 3D pose with velocity and biases), particle filters become computationally prohibitive. Resampling can cause particle depletion, where all particles collapse to a single point. Adaptive resampling techniques help but add complexity. Particle filters also require a good proposal distribution; a poor proposal leads to inefficient sampling.

Deep Learning Limits

Deep learning models require large amounts of labeled training data, which is expensive to collect in robotics. They are also black boxes: it is hard to diagnose why the model failed. They can be brittle to distribution shift: if the test environment differs from training (e.g., new terrain, different sensor calibration), performance can degrade unpredictably. Real-time inference on embedded hardware can be challenging for large models.

Reader FAQ

Which fusion workflow is best for low-cost robots with limited compute?

Kalman filter-based workflows are usually the best choice. They are computationally lightweight, well-understood, and have deterministic runtime. Particle filters require more memory and CPU, and deep learning models often require a GPU. For a low-cost robot with a simple microcontroller, an EKF is the pragmatic choice.

How do I handle sensor dropout in a Kalman filter?

When a sensor drops out, simply skip the update step and let the prediction propagate the state. The covariance will grow, reflecting increased uncertainty. When the sensor returns, you can use a gating test to reject outliers. For long dropouts, consider using a smoother or a robust update that limits the innovation.

Can I combine multiple workflows?

Yes, hybrid approaches are common. For example, use a deep learning model to generate a proposal distribution for a particle filter (a particle filter with a learned proposal), or use a Kalman filter as a baseline and a neural network to predict the innovation. Many production systems use a cascade: a fast Kalman filter for real-time control and a slower particle filter for localization refinement.

What is the most robust workflow for outdoor autonomous navigation?

Robustness depends on the specific obstacles. For environments with frequent GPS dropouts and multimodal sensor data (e.g., urban canyons), a particle filter with a robust likelihood and adaptive resampling often performs well. For smooth, well-modeled dynamics with occasional outliers, an EKF with careful outlier rejection is sufficient. Deep learning is promising but requires extensive validation.

How do I choose the number of particles?

A common rule of thumb is to use at least 100 particles per dimension of the state space. For a 3D pose (x, y, theta), 300–500 particles may suffice. For higher dimensions, adaptive methods like the KLD-sampling (Kullback-Leibler divergence) can adjust the number dynamically. Start with a low number and increase until the estimate converges reliably.

What is the biggest mistake teams make with sensor fusion?

Underestimating the importance of sensor calibration and timing. Even the best fusion workflow will fail if sensors are misaligned or timestamps are inaccurate. Many teams spend months tuning the fusion algorithm only to find that the problem was a 100 ms delay in the lidar driver. Always verify that sensors are synchronized and calibrated before tuning the fusion parameters.