Luca Brembilla

MSc thesis, Politecnico di Milano, 2025

Adaptive LiDAR Cropping

An adaptive front end for streaming LiDAR detection. It projects previous detections into the next frame, crops the input around them, and periodically runs a full scan to recover unseen objects. Developed as my MSc thesis and later published at ICIAP.

This thesis started from a deployment constraint: an Ouster OSDome-128 LiDAR mounted above a road, an NVIDIA Jetson AGX Orin, and 3D detectors trained on other datasets. Retraining the models was not the objective. The question was whether a streaming system had to send the complete point cloud through the detector at every frame.

The proposed front end reuses the detector’s previous output as a spatial prior. It keeps the points around projected object locations and removes the rest before inference. The detector itself remains unchanged.

  • 50.25%of the input points retainedVoxelNeXt, full refresh every 3 frames
  • −3.1 / −1.7mAP points for cars / pedestriansat the same operating point
  • +76%PV-RCNN throughput2.43 to 4.27 Hz on Jetson, 50 W

A detector with memory

For each incoming frame, the system carries forward the boxes and velocities predicted in the previous frame. On a moving sensor, it first compensates for ego motion. It then applies a constant-velocity update, expands each projected box to tolerate prediction error, and keeps only the points that fall inside at least one box.

An object entering outside the current crops could remain unseen. The system therefore processes a complete point cloud at a fixed interval. That refresh frequency and the box expansion factor are the two controls: both exchange computation for recall.

  1. 01DetectRun the unchanged detector on a full point cloud.
  2. 02ProjectMove the detected boxes forward using velocity and ego motion.
  3. 03CropKeep only points inside expanded predicted boxes.
  4. 04UpdateDetect again and use the new output as the next state.
Periodic refresh

A full frame resets the search and allows new objects to enter the state.

The detector is not retrained. The method changes its input from frame to frame.

The implementation performs the point-in-rotated-box test as a vectorized GPU operation. The evaluation covers three existing detectors, but in two separate experiments: SECOND and VoxelNeXt measure the accuracy trade-off on nuScenes Mini; SECOND and PV-RCNN measure throughput on the roadside setup.

How much accuracy is exchanged

  • Car
  • Pedestrian
Detection accuracy against the share of points processed, VoxelNeXt on nuScenes Mini0.650.700.750.800.850.9025%50%75%100%points processedCar: 0.664 mAP at 27% of pointsCar: 0.775 mAP at 32% of pointsCar: 0.826 mAP at 41% of pointsCar: 0.855 mAP at 50% of pointsCar: 0.870 mAP at 62% of pointsCar: 0.886 mAP at 100% of pointsCarPedestrian: 0.853 mAP at 27% of pointsPedestrian: 0.872 mAP at 32% of pointsPedestrian: 0.892 mAP at 41% of pointsPedestrian: 0.892 mAP at 50% of pointsPedestrian: 0.899 mAP at 62% of pointsPedestrian: 0.909 mAP at 100% of pointsPedestrianhalf the points
Show the numbers
Points processedCar mAPPedestrian mAP
100% (baseline)0.8860.909
61.7%0.8700.899
50.3%0.8550.892
40.7%0.8260.892
32.1%0.7750.872
27.2%0.6640.853
VoxelNeXt on nuScenes Mini. Per-class mAP as the retained input falls. Each point uses an expansion factor of 2; the baseline processes the complete cloud.

At the 50.25% operating point, car mAP moves from 0.886 to 0.855 and pedestrian mAP from 0.909 to 0.892. More aggressive cropping is not free. At 27.20% of the points, car mAP falls to 0.664 while pedestrian mAP remains at 0.853. Increasing the expansion factor can recover accuracy, but it also retains more points.

This class difference is an observation from these experiments, not a general property of cropping. Faster motion, detector errors and box geometry all affect how quickly a projected region becomes stale.

When fewer points become faster inference

  • Baseline
  • With cropping
Inference rate on an NVIDIA Jetson AGX Orin, with and without adaptive croppingSECOND (one-stage)0246810Base: 4.87 Hz at 15 W4.87With cropping: 5.93 Hz at 15 W5.9315 WBase: 7.98 Hz at 30 W7.98With cropping: 8.18 Hz at 30 W8.1830 WBase: 9.37 Hz at 50 W9.37With cropping: 8.52 Hz at 50 W8.5250 WPV-RCNN (two-stage)Base: 1.26 Hz at 15 W1.26With cropping: 2.09 Hz at 15 W2.0915 WBase: 1.85 Hz at 30 W1.85With cropping: 3.08 Hz at 30 W3.0830 WBase: 2.43 Hz at 50 W2.43With cropping: 4.27 Hz at 50 W4.2750 WHz
Show the numbers
Power modeSECOND baseSECOND croppedPV-RCNN basePV-RCNN cropped
15 W4.875.931.262.09
30 W7.988.181.853.08
50 W9.378.522.434.27
Custom roadside sequence, NVIDIA Jetson AGX Orin. Effective throughput includes cropping overhead. The adaptive configuration processes one full frame every 10 frames.

PV-RCNN improves in every power mode. At 50 W it moves from 2.43 to 4.27 Hz, a 76% increase. This is a substantial speed-up, but it does not make the detector match the 10 Hz sensor rate.

SECOND exposes the boundary of the method. It improves from 4.87 to 5.93 Hz at 15 W, changes little at 30 W, and becomes slower at 50 W: 8.52 Hz with cropping against 9.37 Hz without it. Input reduction only helps when detector compute is the bottleneck. Otherwise, the cropping pass and lower GPU utilization can outweigh the saved work.

Scope

The custom sequence is small and does not have exhaustive annotations, so its results are limited to throughput and qualitative inspection with KITTI-pretrained models. The constant-velocity projection is appropriate only while the interval between frames and relative motion remain small. A new object can also be detected only on a full refresh or after it enters an existing crop.

The result is therefore not a faster detector architecture. It is a stateful scheduling layer for a streaming detector, useful when the cost saved inside the model is larger than the cost of deciding which points to keep. The work was later developed into an ICIAP paper with Simone Mentasti and Matteo Matteucci.

All projects