When we first prototyped the sensing stack for the Closer unit, we tried ultrasonic-only for about three months. The range was good and the latency was low, but we kept running into edge cases that cost us false negatives in exactly the scenarios that mattered: lateral approaches in narrow aisles, bodies partially occluded by shelving, and slow-moving workers bending down at floor level near the robot's travel path.
When we added depth camera, those gaps largely closed. But depth camera alone introduced a different problem: false positives from reflective floor surfaces and direct overhead lighting, which disrupted operations more than the false negatives from ultrasonic-only had. Neither sensor alone was adequate for mixed-traffic retail and logistics environments. Here is why, and how we currently use both together.
What ultrasonic does well
Ultrasonic proximity sensing works by emitting a sound pulse at 40kHz and measuring the time-of-flight of the return echo. The physics are simple and the latency is deterministic: at 50Hz polling frequency, you get a new range reading every 20ms. The system does not need to process an image or compute a depth map. It reports a distance to the nearest object in the sensor's cone, with no ambient light dependency.
This makes ultrasonic particularly reliable in three aisle scenarios. First, frontal approach detection: an object moving directly toward the sensor face, regardless of lighting. Second, low-light environments like unheated storage aisles where facility lighting is motion-activated. Third, highly textured or transparent surfaces like shrink-wrapped pallet stacks or glass-fronted product displays, which can cause depth cameras to give noisy or absent readings due to reflection and infrared scatter.
The coverage pattern we run uses eight ultrasonic sensors arranged in an octagonal ring around the chassis, spaced to give overlapping coverage zones. This gives 360-degree detection capability for objects at height between 30cm and 120cm from floor level, up to a range of approximately 1.5m. The overlap between adjacent sensors means an object passing through the gap between two sensors is caught by both, with the closer reading used for the proximity calculation.
What ultrasonic handles poorly
Ultrasonic has two significant failure modes in aisle environments. The first is angular approach: an object approaching the robot from the side, where the sonic cone is oriented tangentially rather than perpendicular to the approach vector. Sound pulses return weakly when the reflecting surface is angled more than about 45 degrees from perpendicular to the beam. A person stepping out sideways from a rack gap at 90 degrees to the robot's travel path will produce a weak or absent ultrasonic return until they are nearly within the robot's chassis boundary.
The second failure mode is above-floor-level differentiation. Ultrasonic reports "nearest object at X distance" but cannot distinguish between a human leg at 0.8m and a dangling price tag at 0.2m. Low-hanging merchandising signage in retail environments caused persistent false positive stops in our first retail deployment test. We had to add a minimum-height filter in firmware, which then introduced blind spots for short obstacles. This is a structural tension in ultrasonic-only configurations that does not have a clean firmware solution.
What depth camera adds
The depth camera generates a per-pixel distance map of the scene ahead of the robot at 30fps. Rather than a single range scalar like ultrasonic, it gives a spatial model: you can see where in 3D space objects are, how tall they are, and roughly what shape they present. This solves the angular approach problem directly. A body stepping out from a rack gap appears as a sudden depth discontinuity across a cluster of pixels in the frame. The detection algorithm looks for point cloud changes above a threshold size that exceeds noise, which filters the price-tag false positive problem that plagued ultrasonic.
The depth camera we use is a structured light stereo unit rather than a time-of-flight camera. Time-of-flight cameras are faster and work at longer range, but they are susceptible to interference from other time-of-flight emitters in environments with multiple AMRs, and structured light handles direct-sunlight-adjacent environments better for our specific use case. The depth camera runs at 30fps, which is sufficient for the motion speeds in typical retail and logistics aisles. For a robot moving at 1.2 m/s with a detection range of 1.5m, 30fps gives you between 37 and 42 valid frames to detect and respond before the closest point of approach from a stationary detection boundary, which is more than adequate.
The failure modes depth camera introduces
Structured light depth cameras struggle with highly reflective floor surfaces. Polished concrete, wet floor sections, and metal anti-fatigue matting all create specular reflections that appear in the depth map as phantom objects at floor level. Our first firmware build triggered protective stops roughly 3 times per hour on a facility with high-gloss resin-coated floors. That is not a viable operation.
The mitigation is a floor exclusion zone in the depth map: we mask out pixels within a defined height band from the floor plane, calculated dynamically using the robot's odometry and IMU to maintain an accurate floor model even as the floor surface changes across zones. This effectively removes floor-level noise from the detection input. The tradeoff is that we lose some detection sensitivity for low-height obstacles below approximately 25cm floor clearance, which is why the third sensing layer (floor-contact force feedback) exists.
Depth cameras also degrade under direct overhead LED lighting when the structured light pattern washes out against bright reflective surfaces. In the early prototype, this caused detection dropouts that we could reproduce reliably in one aisle of a retail test site with high-intensity LED shelf lighting. The fix was adjusting the structured light projector intensity and adding a confidence threshold filter: pixel depth readings below a confidence score are treated as absent rather than zero-distance. Detection reliability with this filter in place has been solid across 14 different lighting configurations we have tested.
How sensor fusion actually works in practice
The fusion logic is not symmetrical. The two sensors are not averaged or weighted by a static formula. They feed separate detection pipelines that each produce a binary "obstacle detected" signal and a distance estimate. The safety controller acts on whichever pipeline produces a detection event first, treating it as ground truth.
A confirmed detection from either sensor alone is enough to trigger deceleration. A confirmed detection from both sensors within one control loop tick triggers an emergency stop rather than a gradual deceleration, on the assumption that double-confirmation of an obstacle in close proximity warrants a harder response. This two-tier response is tunable per deployment: operators with higher cargo stability requirements can set both tiers to gradual deceleration, accepting slightly larger stopping distances in exchange for less abrupt motion.
False positive management is asymmetric too. A false positive from ultrasonic (brief erroneous stop) is operationally disruptive but physically safe. A false negative from either sensor (failure to detect a person) is the failure mode we design against with redundancy. This is why we do not "vote" between sensors or require both to agree before acting. Requiring agreement would reduce false positives at the cost of potentially requiring both sensors to fail simultaneously before triggering a protective stop. We accept the false positive risk to eliminate the false negative risk.
When one layer genuinely outperforms the other by scenario
Based on our testing and pilot data, depth camera outperforms ultrasonic in three scenarios: lateral approach from rack gaps, distinguishing above-floor obstacles from low-hanging signage, and crowded aisle scenarios where multiple people are present and ultrasonic may not resolve individual bodies distinctly from the aggregate return.
Ultrasonic outperforms depth camera in three scenarios: low-light or dark aisles, highly reflective floor surfaces before the floor exclusion filter kicks in, and transparent or semi-transparent obstacle surfaces like shrink-wrap where structured light scatters rather than reflects cleanly.
If we had to ship with only one sensor type, we would choose depth camera for the majority of retail and logistics environments we target. But the scenarios where depth camera struggles are exactly the scenarios where failures are costly, which is why removing the ultrasonic layer is not a cost reduction we would make. The redundancy is not a belt-and-suspenders conservatism. It is an engineering decision based on which failure modes we can and cannot tolerate.
