LiDAR-Camera Calibration Validator

A ROS1 / ROS2 toolkit for validating LiDAR-camera calibration through real-time projection, visual overlay, and field-oriented sensor synchronization.

ROS1 repository / ROS2 repository

Overview

LiDAR-camera calibration is a small but important part of robotic perception. A calibration result may look numerically valid, yet still fail visually when projected into real scenes. This project focuses on fast visual validation: projecting LiDAR point clouds onto camera images and helping users judge whether the extrinsic parameters are actually usable.

The validator is designed as a practical tool rather than a full calibration algorithm. It helps inspect, compare, and refine existing calibration parameters through visualization, online parameter tuning, and lightweight quantitative cues. Later versions also target practical deployment issues: non-repetitive LiDAR scans, missing hardware time synchronization, ROS2 parameter workflows, and QoS compatibility.

System pipeline for LiDAR-Camera Calibration Validator — Projection and validation pipeline.

Features

Distortion-aware point cloud projection onto camera images.
Visual checking of LiDAR-camera alignment through projected points.
Edge-overlap and lightweight image-depth consistency metrics for calibration quality estimation.
ApproximateTime and latest-style synchronization modes for both time-aligned sensors and loosely synchronized field setups.
Multi-frame point cloud accumulation for sparse or non-repetitive LiDARs such as Livox-style sensors.
Online parameter adjustment through ROS1 dynamic_reconfigure or ROS2 native parameters.
ROS2 support with configurable fused-image QoS, render timer, projection cache, metrics interval, and TBB-based projection for larger point clouds.

Visualization GUI for calibration validation — Visualization and online parameter adjustment.

Evolution

The project gradually moved from a basic ROS1 visual overlay into a field-oriented validation tool. The first ROS1 version started with projected point-cloud visualization and later added distortion-corrected projection, edge-based metrics, dynamic parameter tuning, FPS and processing-time monitoring, and parallel projection for large point clouds.

The next stage focused on real LiDAR deployment. It added support for non-repetitive LiDAR scans, especially Livox-style sensors, and introduced latest/latest_pair synchronization so visual validation can still work when strict hardware time synchronization is unavailable.

The ROS2 Humble implementation then moved the build and launch path to ament_cmake, rclcpp, and ros2 launch; runtime tuning moved from dynamic_reconfigure to ROS2 parameters; the fused image publisher gained configurable QoS reliability; and parameter updates are checked after application so tuning failures are easier to notice.

Livox LiDAR validation example — Validation example with Livox LiDAR-camera projection.

Why It Matters

Multi-sensor perception pipelines depend heavily on reliable extrinsic calibration. When the alignment between LiDAR and camera is off, downstream perception modules may silently inherit geometric errors. A lightweight validation tool makes this failure mode easier to notice before it affects mapping, detection, tracking, or planning.

The project is also shaped by field constraints. Spinning LiDAR, non-repetitive LiDAR scans, and Livox-style sensors do not always produce equally dense or equally regular point clouds, and not every setup has precise hardware time synchronization. Supporting accumulation, latest-style matching, online tuning, and configurable ROS2 QoS makes the validator useful outside carefully prepared lab recordings.

Principle

The validator treats calibration quality as a LiDAR-to-image projection problem. It loads three fields from an OpenCV YAML calibration file: the camera intrinsic matrix K_0, the distortion coefficients C_0, and the LiDAR-to-camera extrinsic transform E_0.

E₀ = ∈ SE(3), R ∈ SO(3), t ∈ R³
p_c = E₀p_l, p_l = [x_l, y_l, z_l, 1]^T, p_c = [X_c, Y_c, Z_c, 1]^T E₀ is expected to map LiDAR to camera. If a calibration tool exports camera-to-LiDAR, the inverse transform should be used. Only points with positive camera-frame depth above a small near-plane threshold are valid for projection.

Each incoming PointCloud2 frame is converted to a PCL cloud, optionally accumulated, downsampled, and distance-filtered. Valid points are normalized by depth, corrected with the Brown-Conrady radial-tangential distortion model when distortion parameters are available, and mapped to image pixels through the camera intrinsics.

x_n = X_c / Z_c, y_n = Y_c / Z_c, r² = x_n² + y_n² x_d = x_n(1 + k₁r² + k₂r⁴ + k₃r⁶) + 2p₁x_ny_n + p₂(r² + 2x_n²) y_d = y_n(1 + k₁r² + k₂r⁴ + k₃r⁶) + p₁(r² + 2y_n²) + 2p₂x_ny_n K₀ = u = f_xx_d + c_x, v = f_yy_d + c_y C₀ provides k₁, k₂, p₁, p₂, and optionally k₃. The projected pixel is drawn only if it falls inside the image.

The overlay is paired with lightweight quality cues. Projection ratio measures how many LiDAR points land inside the image. Edge overlap runs Canny edge detection and a distance transform, then checks how many projected points fall near visual edges. The image-depth consistency score in this implementation samples image intensity at projected pixels and compares it with point depth as a correlation-style signal.

R_proj = N_valid / N_total, S_edge = (1 / N) Σ_i=1^N 1[D(u_i, v_i) < τ]
S_corr = | cov(I(u_i, v_i), Z_i) | / (σ_Iσ_Z) These scores give quick cues, while the visual overlay remains the main validation signal.

The two versions show the same validation idea across two ROS generations: a ROS1 implementation focused on dynamic_reconfigure and synchronized visual inspection, and a ROS2 implementation redesigned around parameters, QoS, projection caching, render scheduling, and TBB-based parallel processing for larger point clouds.