Obstacle avoidance metrics beyond collision counts

"Zero collisions in testing" is the most common — and least informative — safety claim in mobile robotics. It can describe a robot that navigated cleanly, or a robot that never moved, or a robot that brushed walls because the bumper absorbed every contact, or a robot that deadlocked in front of the first obstacle until the test ended.

Collision counts collapse all of that into one boolean. The metric stack that actually distinguishes safe navigation from lucky navigation measures what happened around collisions: proximity, availability, and recovery.

Why collision counts are not safety

A collision counter is a binary event detector with three blind spots:

• Near-misses are invisible. A robot that passes within 2cm of a person at speed and one that keeps 1m clearance both report zero collisions. The safety-relevant difference is entirely in the unrecorded distribution.
• Non-collision failures are invisible. Freezing in place until timeout, oscillating in front of an obstacle, or deadlocking at a corridor corner all produce "no collisions" and no progress. These are failures of the same system — avoidance — but they never touch the counter.
• Contact quality is invisible. A 0.1 m/s bumper graze and a 1.5 m/s frontal impact both increment the counter by one. The injury physics differ by orders of magnitude.

The navigation evaluation post covers why success rate hides localization and obstacle-handling problems; collision counting is the same blind spot in smaller packaging.

The avoidance metric stack

1. Clearance distribution

Replace the collision boolean with the distribution of minimum clearance over time — distance to nearest obstacle per timestep, aggregated as percentile values. p5 clearance, mean clearance, and the fraction of time below a safety threshold describe how the robot actually moves through the world. Near-miss events become countable instead of invisible.

2. Progress-constrained success

Success should not be "reached goal without collision"; it should be "reached goal within a time budget without collision and without violating clearance thresholds." This single constraint kills the freeze-and-wait exploit: a robot that hides until the test ends now fails. Progress (distance-to-goal reduction per second) should be reported as its own series.

3. Failure taxonomy for non-collisions

Label and count the ways avoidance fails without contact:

• Deadlock: no progress for N seconds with an obstacle in the path.
• Oscillation: heading flips repeatedly at an obstacle boundary.
• Brushing: sustained sub-threshold clearance.
• Timeouts: goal unreached within the budget.

Each is a different bug with a different fix, and each disappears inside a collision count. The same taxonomy discipline applies in manipulation — see manipulation metrics that survive a skeptic.

4. Recovery and near-miss response

When a near-miss happens, what does the robot do? Slow down? Replan? Brake? Report reaction latency and speed reduction after clearance drops below threshold. Reactive competence is the difference between a planner and a safe planner.

5. Perturbation curves

The avoidance stack only earns trust under stress. Sensor dropout, obstacle speed variation, dynamic pedestrians, degraded localization — run the same metrics under perturbation and report the degradation. Static maps hide all of this; a clean-lab success rate is the least predictive number in the stack.

Reporting that survives review

An avoidance evaluation worth citing reports:

1. clearance percentiles (not just minimums),
2. progress-constrained success with time budget,
3. non-collision failure rates by class,
4. near-miss reaction metrics,
5. perturbation curves with intervals.

None of this is exotic. All of it is currently missing from most vendor claims. When a robot vendor says "we never collide," the correct reply is: "Show me the clearance distribution, the deadlock rate, and the behavior under sensor dropout." If the answer is another video of an empty corridor, the claim is marketing.

Mobile navigation is the evaluation surface where the gap between demo evidence and deployment evidence is widest — which is exactly why it needs the strictest protocol discipline. The sim-to-real benchmark design guide shows how to build that protocol from scratch.

A worked example: the corridor test

To see the stack in action, evaluate a mobile robot on a 4m corridor with a stationary obstacle at 2m, repeated over 20 trials with pinned seeds.

• Collision-only result: 20 trials, 0 collisions. "Perfect."
• Clearance distribution: the robot passes the obstacle with a median clearance of 0.3m but spends 12% of the approach below the 0.4m safety threshold, brushing to 0.15m at p5. The near-miss is invisible to the collision counter — and it is the operationally relevant number for a facility with shelves or people.
• Progress-constrained success: 18 of 20 trials reach the goal within the 30s budget. Two "succeed" only after 55s and 48s — the robot crept past the obstacle at 0.05 m/s. With a progress floor (distance-to-goal reduction ≥ 0.3 m/s averaged), both are failures: the robot did not collide, but it also did not navigate.
• Non-collision failures: of the two slow trials, one shows clear oscillation — heading flips every ~1.5s at the obstacle boundary for 20s before committing. Classified as oscillation, not "success."
• Near-miss response: on trials where clearance drops below 0.3m, reaction latency averages 0.8s and speed reduction averages 40%. One trial shows no speed response at all — clearance 0.15m at full speed.

The aggregate story was "perfect"; the layered story is "safe in aggregate, marginal in clearance, weak in progress discipline, one oscillation pattern, one blind near-miss response." Those are five different findings for five different fixes — all hidden by the boolean.

Sensor dropout and dynamic obstacles

Static-obstacle tests are the easy half. The metric stack earns its keep when the world moves, and two conditions belong in every navigation evaluation:

Sensor dropout tiers. Lidar/camera dropout is the most common real-world failure, and it is almost never tested. Run the full clearance/progress/recovery stack under tiered dropout — 0%, 10%, 25%, 50% of frames dropped, deterministic patterns with pinned seeds. The degradation curve is the finding: a robot that holds clearance through 25% dropout and collapses at 50% is a different product from one that degrades linearly from 10%.

Dynamic obstacles. A pedestrian crossing the path, an obstacle appearing after the plan is formed, a door opening into the corridor. Score the same stack: clearance percentiles during the encounter, reaction latency, speed reduction, and whether recovery replans or deadlocks. The navigation evaluation post covers why clean-map success hides these brittleness points; the dynamic conditions are where they surface.

Both conditions belong in the frozen protocol — chosen and versioned before the first run, with intervals reported per condition. That is the discipline that turns "we never collide" into a claim with a shape a reviewer can check.