The Big Picture
Linking an instrumented real vehicle with a matched virtual twin lets teams test shared sensing end-to-end: adding cooperating agents widens coverage and restores detection in poor conditions, but large positioning errors (around 1.5–2.0 m) erase those benefits.
ON THIS PAGE
The Evidence
Cooperation between vehicles raises the shared field of view from roughly one quarter (single vehicle) to a clear majority when all agents participate, and it improves obstacle recall most when individual sensing is weakest (night). The occupancy-grid fusion stays stable across realistic localization noise up to a point, but at about 2.0 meters of position error all cooperative setups converge to similar recall, meaning localization uncertainty dominates. The deployed setup is a working hybrid testbed: a real instrumented car is reflected in a calibrated virtual twin, live messages feed a unified perception module, and the system runs across nominal, rain and night conditions. cooperative setups
Data Highlights
1Joint field-of-view coverage rises from ≈25% with a single vehicle to a majority (>50%) with all agents sharing data.
2Localization-noise tested at 0, 0.5, 1.0, 1.5 and 2.0 meters — at 2.0 m occupied-cell recall for different cooperation levels converges, showing position error dominates.
3Cooperative messages (position and detections) are exchanged every 100 ms and feed an occupancy grid evaluated as area-under-curve across three weather conditions (nominal, rain, night).
What This Means
Engineers building cooperative perception and vehicle testbeds will get a deployable recipe for coupling a real vehicle to a virtual twin and for exercising multi-agent scenarios. Test managers and safety leads at suppliers can use the platform to scale tests that would be infeasible with only physical vehicles and to evaluate how more agents help in adverse conditions. multi-agent fleet management
Not sure where to start?Get personalized recommendations
Key Figures
Fig 1: Figure 1: Twin-construction pipeline: (a) the source road network in OpenStreetMap is (b) converted to an OpenDRIVE (.xodr) logical map and (c) imported into CARLA as a drivable 3D scenario.
Fig 2: Figure 2: Custom twin assets. Top: the real vehicle is reconstructed as a 3D mesh and imported as a calibrated, instrumented CARLA model. Bottom: custom terrain, buildings and vegetation (left) and retroreflective cat’s-eye road studs under night lighting (right).
Fig 4: Figure 4: Left: drone footage of the real double T-intersection. Right: CARLA DT reconstruction. Center bottom: occupancy grid produced from the live CAM/CPM stream (with one virtual agent added).
Fig 5: Figure 5: Top (a): Joint FoV coverage of CP across conditions. Bottom (b): Occupied-cell recall across different environmental conditions and localization noise.
Ready to evaluate your AI agents?
Learn how ReputAgent helps teams build trustworthy AI through systematic evaluation.
Learn MoreConsiderations
Quantitative sim-to-real fidelity (how closely virtual results match physical ground truth) remains work in progress; current metrics are measured inside the virtual twin with controlled synthetic noise rather than directly against real-vehicle ground truth. The CARLA-based co-simulation can introduce timing jitter that affects determinism; improving and bounding the ingest-update loop is an engineering priority. The demonstrated gains depend on reasonable localization quality — if real-world positioning errors exceed about 1.5–2 meters, cooperation no longer improves recall significantly. ground truth
Methodology & More
The deployed platform couples one instrumented real vehicle (centimetre-level differential GPS optional, LiDAR, camera, on-board compute, and V2X connectivity) with a detailed virtual twin built from map data and augmented assets (vehicle 3D model, buildings, vegetation, night reflectors). Live cooperative messages (state and detections) are parsed into the simulator and into virtual agents so that real and simulated participants jointly feed a ROS-based perception stack. The cooperative perception module fuses messages into a probabilistic occupancy grid (regen every batch), implemented with JAX on GPU and tested unchanged on both the real vehicle and the digital twin. Evaluation ran a real-world demonstrator on a test track and a controlled set of simulations across three environmental conditions (nominal, rain, night), three cooperation configurations (ego-only, three connected agents, all six agents), and five localization-noise levels (0–2.0 m). Results show cooperation substantially widens coverage and improves obstacle recall, especially in night conditions where single-vehicle sensing is weakest. However, localization uncertainty becomes the dominant failure mode above roughly 1.5–2.0 m, at which point additional agents no longer raise recall. Engineering next steps include raising and bounding the loop rate to ensure repeatability, benchmarking GPU versus CPU for the occupancy module, running a measured sim-to-real fidelity assessment against differential-GPS ground truth, and expanding to a Mediterranean-focused scenario library for broader testing needs. cooperation configurations localization-noise levels
Avoid common pitfallsLearn what failures to watch for
Credibility Assessment:
ArXiv preprint with no listed affiliations and low author h-indices (max h≈7). Limited recognizable institutional signals — fits 'emerging/limited info'.