AutoNav Showcase (static, read-only)

E · Benchmark and metrics explorer

Metrics

Latency is wall-clock per frame, pooled over repeats. The dashed line is the sensor period. A run's hardware is in its label.

Read the spread, not just the median. These runs were measured on a shared development laptop (AC power, Balanced power plan, about 66 % processor performance, 1.9–2.7 GB RAM free), not an isolated benchmark host. Repeat medians of the same variant differ by up to 2.4×, and baseline medians ranged 650–890 ms across sessions. Runs are labelled with their commit; “uncommitted changes” means the working tree was dirty when the run started. Profiles and variants were measured at different times, so only differences larger than the per-repeat spread are claims. Background load was recorded only for the later runs.
RunFramesP50P90P95P99MaxViolationsAcceptedATE RMSEPeak RSSEvidence
baseline · laptop-full
newer_college_bag9 · phase3 · 0862f4f · uncommitted changes
270 749.9 ms 1275.7 ms 1547.6 ms 2111.6 ms 2684.2 ms 100.0 % 100.0 % 0.51 m 77 MB REAL
baseline · laptop-4c
newer_college_bag9 · phase3 · 0862f4f · uncommitted changes
270 676.2 ms 1104.0 ms 1309.6 ms 1816.2 ms 2328.2 ms 100.0 % 100.0 % 0.51 m 78 MB REAL
baseline · laptop-1c
newer_college_bag9 · phase3 · 0862f4f · uncommitted changes
270 671.5 ms 1108.8 ms 1367.2 ms 1864.3 ms 2318.7 ms 100.0 % 100.0 % 0.51 m 75 MB REAL
p2plane · laptop-full
newer_college_bag9 · phase5 · 5539c47 · uncommitted changes
270 734.2 ms 921.7 ms 996.8 ms 1232.5 ms 1770.1 ms 100.0 % 100.0 % 0.15 m 97 MB REAL
baseline · laptop-full
oxford_spires · phase5 · 5539c47 · uncommitted changes
300 522.3 ms 806.1 ms 921.5 ms 1294.8 ms 2433.4 ms 100.0 % 100.0 % 1.32 m 75 MB REAL
baseline · laptop-4c
oxford_spires · phase5 · 5539c47 · uncommitted changes
300 437.8 ms 644.1 ms 756.7 ms 1067.9 ms 1720.4 ms 100.0 % 100.0 % 1.32 m 74 MB REAL
baseline · laptop-1c
oxford_spires · phase5 · 5539c47 · uncommitted changes
300 779.5 ms 1466.3 ms 2103.4 ms 2936.4 ms 4978.1 ms 100.0 % 100.0 % 1.32 m 74 MB REAL
p2plane · laptop-full
oxford_spires · phase5 · 5539c47 · uncommitted changes
300 517.6 ms 619.4 ms 657.8 ms 794.0 ms 2876.8 ms 100.0 % 100.0 % 0.15 m 86 MB REAL
optimized · laptop-full
newer_college_bag9 · phase5-v2 · 2917b44
270 323.5 ms 765.8 ms 1017.9 ms 1520.8 ms 3190.3 ms 97.2 % 100.0 % 0.51 m 80 MB REAL
optimized · laptop-4c
newer_college_bag9 · phase5-v2 · 2917b44
270 611.4 ms 1349.4 ms 1746.0 ms 2693.6 ms 6835.5 ms 99.9 % 100.0 % 0.51 m 76 MB REAL
optimized · laptop-1c
newer_college_bag9 · phase5-v2 · 2917b44
270 869.8 ms 1633.4 ms 2090.3 ms 2808.4 ms 4922.3 ms 99.9 % 100.0 % 0.51 m 76 MB REAL
optimized · laptop-full
oxford_spires · phase5-v2 · 2917b44 · uncommitted changes
300 247.5 ms 375.2 ms 456.7 ms 1055.6 ms 3694.6 ms 98.8 % 100.0 % 1.32 m 76 MB REAL
baseline · laptop-full
newer_college_bag9 · final · e9036cd
270 887.5 ms 1602.3 ms 2069.0 ms 4124.3 ms 8970.7 ms 100.0 % 100.0 % 0.51 m 78 MB REAL
optimized · laptop-full
newer_college_bag9 · final · e9036cd
270 505.8 ms 1803.3 ms 2301.0 ms 3642.5 ms 10755.3 ms 98.8 % 100.0 % 0.51 m 79 MB REAL

Single-frame preprocessing (NON-CONSECUTIVE frames)

Median 143 ms, P95 186 ms per frame for invalid-point removal, range filter and voxel downsample; voxel stage alone 133 ms; 29918 points after voxel filter (median). No odometry or accuracy claim.


Scaling across hardware profiles

newer_college_bag9 · baseline

  • laptop-full vs laptop-4c: per-repeat median ranges overlap (674-839 ms vs 649-739 ms); no difference is claimed.
  • laptop-4c vs laptop-1c: per-repeat median ranges overlap (649-739 ms vs 650-676 ms); no difference is claimed.
  • Profiles were measured at different times on a shared, non-isolated laptop; a non-monotonic ordering across profiles reflects machine drift as much as core count.
  • [ASSUMED, one-way inference, not measured] It misses the deadline on 4 laptop cores, so a Raspberry Pi 5 very likely misses it too.

oxford_spires · baseline

  • laptop-4c is faster than laptop-full beyond run-to-run spread (median 438 ms vs 522 ms).
  • laptop-4c is faster than laptop-1c beyond run-to-run spread (median 438 ms vs 779 ms).
  • Profiles were measured at different times on a shared, non-isolated laptop; a non-monotonic ordering across profiles reflects machine drift as much as core count.
  • [ASSUMED, one-way inference, not measured] It misses the deadline on 4 laptop cores, so a Raspberry Pi 5 very likely misses it too.

newer_college_bag9 · optimized

  • laptop-full is faster than laptop-4c beyond run-to-run spread (median 324 ms vs 611 ms).
  • laptop-4c is faster than laptop-1c beyond run-to-run spread (median 611 ms vs 870 ms).
  • Profiles were measured at different times on a shared, non-isolated laptop; a non-monotonic ordering across profiles reflects machine drift as much as core count.
  • [ASSUMED, one-way inference, not measured] It misses the deadline on 4 laptop cores, so a Raspberry Pi 5 very likely misses it too.

Verdicts

baseline · laptop-full · newer_college_bag9

Did the pipeline meet the 100 ms deadline?

FAIL

No. 100.0 % of 798 frames missed 100 ms. Median latency 750 ms is 7.5x the deadline; P99 is 2112 ms.

Rule and evidence

pass: 0 violations; warn: 0 < violations <= 5 %; fail: violations > 5 %. Violation = per-frame pipeline latency (preprocess + register) strictly greater than the deadline.

{
  "deadline_ms": 100,
  "p50_ms": 749.8913,
  "p95_ms": 1547.597945,
  "p99_ms": 2111.641545999998,
  "max_ms": 2684.1567,
  "violation_percent": 100,
  "frames_measured": 798
}
Did registration produce valid matches?

PASS

269 of 269 registrations passed the validity gate (100.0 %).

Rule and evidence

pass: >= 99 % accepted; warn: >= 90 %; fail: < 90 %. Accepted = passes the registration validity gate.

{
  "attempts": 269,
  "accepted": 269,
  "rejected": 0,
  "rejected_by_reason": {},
  "icp_iterations_median": 10,
  "inlier_ratio_median": 0.99936,
  "residual_median_m": 0.12463
}
How accurate was the estimated trajectory?

INFO

ATE RMSE 0.51 m (max 1.06 m) over 22.5 m of ground-truth path after rigid alignment; position RPE is reported per window in the summary. Both include any unmodelled sensor-to-body offset (lever arm) between the estimate and the ground-truth body frame, so they mix drift with extrinsic bias. Scan-to-scan odometry has no loop closure, so error accumulates with distance.

Rule and evidence

descriptive; no pass/fail threshold is applied to accuracy

{
  "ate_rmse_m": 0.5068686215458409,
  "ate_max_m": 1.0565121346857196,
  "gt_path_length_m": 22.472984119767467,
  "ate_rmse_percent_of_path": 2.2554575700518296,
  "final_error_m": 0.19201671534112752,
  "rpe_mode": "world_position_after_umeyama_alignment",
  "rpe": {
    "time_1s": 28.56793327591613,
    "time_5s": 14.567642505063228,
    "distance_5m": 10.553605423564546,
    "distance_10m": 7.834970613283673
  }
}

baseline · laptop-4c · newer_college_bag9

Did the pipeline meet the 100 ms deadline?

FAIL

No. 100.0 % of 798 frames missed 100 ms. Median latency 676 ms is 6.8x the deadline; P99 is 1816 ms.

Rule and evidence

pass: 0 violations; warn: 0 < violations <= 5 %; fail: violations > 5 %. Violation = per-frame pipeline latency (preprocess + register) strictly greater than the deadline.

{
  "deadline_ms": 100,
  "p50_ms": 676.2093500000001,
  "p95_ms": 1309.6312149999999,
  "p99_ms": 1816.2018479999997,
  "max_ms": 2328.1782,
  "violation_percent": 100,
  "frames_measured": 798
}
Did registration produce valid matches?

PASS

269 of 269 registrations passed the validity gate (100.0 %).

Rule and evidence

pass: >= 99 % accepted; warn: >= 90 %; fail: < 90 %. Accepted = passes the registration validity gate.

{
  "attempts": 269,
  "accepted": 269,
  "rejected": 0,
  "rejected_by_reason": {},
  "icp_iterations_median": 10,
  "inlier_ratio_median": 0.99936,
  "residual_median_m": 0.12463
}
How accurate was the estimated trajectory?

INFO

ATE RMSE 0.51 m (max 1.06 m) over 22.5 m of ground-truth path after rigid alignment; position RPE is reported per window in the summary. Both include any unmodelled sensor-to-body offset (lever arm) between the estimate and the ground-truth body frame, so they mix drift with extrinsic bias. Scan-to-scan odometry has no loop closure, so error accumulates with distance.

Rule and evidence

descriptive; no pass/fail threshold is applied to accuracy

{
  "ate_rmse_m": 0.5068686215458409,
  "ate_max_m": 1.0565121346857196,
  "gt_path_length_m": 22.472984119767467,
  "ate_rmse_percent_of_path": 2.2554575700518296,
  "final_error_m": 0.19201671534112752,
  "rpe_mode": "world_position_after_umeyama_alignment",
  "rpe": {
    "time_1s": 28.56793327591613,
    "time_5s": 14.567642505063228,
    "distance_5m": 10.553605423564546,
    "distance_10m": 7.834970613283673
  }
}

baseline · laptop-1c · newer_college_bag9

Did the pipeline meet the 100 ms deadline?

FAIL

No. 100.0 % of 798 frames missed 100 ms. Median latency 672 ms is 6.7x the deadline; P99 is 1864 ms.

Rule and evidence

pass: 0 violations; warn: 0 < violations <= 5 %; fail: violations > 5 %. Violation = per-frame pipeline latency (preprocess + register) strictly greater than the deadline.

{
  "deadline_ms": 100,
  "p50_ms": 671.51435,
  "p95_ms": 1367.2325399999995,
  "p99_ms": 1864.297877,
  "max_ms": 2318.7079,
  "violation_percent": 100,
  "frames_measured": 798
}
Did registration produce valid matches?

PASS

269 of 269 registrations passed the validity gate (100.0 %).

Rule and evidence

pass: >= 99 % accepted; warn: >= 90 %; fail: < 90 %. Accepted = passes the registration validity gate.

{
  "attempts": 269,
  "accepted": 269,
  "rejected": 0,
  "rejected_by_reason": {},
  "icp_iterations_median": 10,
  "inlier_ratio_median": 0.99936,
  "residual_median_m": 0.12463
}
How accurate was the estimated trajectory?

INFO

ATE RMSE 0.51 m (max 1.06 m) over 22.5 m of ground-truth path after rigid alignment; position RPE is reported per window in the summary. Both include any unmodelled sensor-to-body offset (lever arm) between the estimate and the ground-truth body frame, so they mix drift with extrinsic bias. Scan-to-scan odometry has no loop closure, so error accumulates with distance.

Rule and evidence

descriptive; no pass/fail threshold is applied to accuracy

{
  "ate_rmse_m": 0.5068686215458409,
  "ate_max_m": 1.0565121346857196,
  "gt_path_length_m": 22.472984119767467,
  "ate_rmse_percent_of_path": 2.2554575700518296,
  "final_error_m": 0.19201671534112752,
  "rpe_mode": "world_position_after_umeyama_alignment",
  "rpe": {
    "time_1s": 28.56793327591613,
    "time_5s": 14.567642505063228,
    "distance_5m": 10.553605423564546,
    "distance_10m": 7.834970613283673
  }
}

p2plane · laptop-full · newer_college_bag9

Did the pipeline meet the 100 ms deadline?

FAIL

No. 100.0 % of 798 frames missed 100 ms. Median latency 734 ms is 7.3x the deadline; P99 is 1232 ms.

Rule and evidence

pass: 0 violations; warn: 0 < violations <= 5 %; fail: violations > 5 %. Violation = per-frame pipeline latency (preprocess + register) strictly greater than the deadline.

{
  "deadline_ms": 100,
  "p50_ms": 734.24235,
  "p95_ms": 996.8155599999999,
  "p99_ms": 1232.468592,
  "max_ms": 1770.1241,
  "violation_percent": 100,
  "frames_measured": 798
}
Did registration produce valid matches?

PASS

269 of 269 registrations passed the validity gate (100.0 %).

Rule and evidence

pass: >= 99 % accepted; warn: >= 90 %; fail: < 90 %. Accepted = passes the registration validity gate.

{
  "attempts": 269,
  "accepted": 269,
  "rejected": 0,
  "rejected_by_reason": {},
  "icp_iterations_median": 5,
  "inlier_ratio_median": 0.99932,
  "residual_median_m": 0.1277
}
How accurate was the estimated trajectory?

INFO

ATE RMSE 0.15 m (max 0.35 m) over 22.5 m of ground-truth path after rigid alignment; position RPE is reported per window in the summary. Both include any unmodelled sensor-to-body offset (lever arm) between the estimate and the ground-truth body frame, so they mix drift with extrinsic bias. Scan-to-scan odometry has no loop closure, so error accumulates with distance.

Rule and evidence

descriptive; no pass/fail threshold is applied to accuracy

{
  "ate_rmse_m": 0.15113568120206913,
  "ate_max_m": 0.3455220968441755,
  "gt_path_length_m": 22.472984119767467,
  "ate_rmse_percent_of_path": 0.6725216393008022,
  "final_error_m": 0.13946928656653032,
  "rpe_mode": "world_position_after_umeyama_alignment",
  "rpe": {
    "time_1s": 5.694847694836872,
    "time_5s": 3.0709409587105023,
    "distance_5m": 2.9858696453402276,
    "distance_10m": 2.528365538024447
  }
}

baseline · laptop-full · oxford_spires

Did the pipeline meet the 100 ms deadline?

FAIL

No. 100.0 % of 888 frames missed 100 ms. Median latency 522 ms is 5.2x the deadline; P99 is 1295 ms.

Rule and evidence

pass: 0 violations; warn: 0 < violations <= 5 %; fail: violations > 5 %. Violation = per-frame pipeline latency (preprocess + register) strictly greater than the deadline.

{
  "deadline_ms": 100,
  "p50_ms": 522.26365,
  "p95_ms": 921.46106,
  "p99_ms": 1294.765847,
  "max_ms": 2433.4405,
  "violation_percent": 100,
  "frames_measured": 888
}
Did registration produce valid matches?

PASS

299 of 299 registrations passed the validity gate (100.0 %).

Rule and evidence

pass: >= 99 % accepted; warn: >= 90 %; fail: < 90 %. Accepted = passes the registration validity gate.

{
  "attempts": 299,
  "accepted": 299,
  "rejected": 0,
  "rejected_by_reason": {},
  "icp_iterations_median": 10,
  "inlier_ratio_median": 0.99795,
  "residual_median_m": 0.15885
}
How accurate was the estimated trajectory?

INFO

ATE RMSE 1.32 m (max 2.79 m) over 45.6 m of ground-truth path after rigid alignment; position RPE is reported per window in the summary. Both include any unmodelled sensor-to-body offset (lever arm) between the estimate and the ground-truth body frame, so they mix drift with extrinsic bias. Scan-to-scan odometry has no loop closure, so error accumulates with distance.

Rule and evidence

descriptive; no pass/fail threshold is applied to accuracy

{
  "ate_rmse_m": 1.3217205014603846,
  "ate_max_m": 2.791091479410538,
  "gt_path_length_m": 45.563618073800534,
  "ate_rmse_percent_of_path": 2.900824292134924,
  "final_error_m": 1.2982540736844619,
  "rpe_mode": "world_position_after_umeyama_alignment",
  "rpe": {
    "time_1s": 15.048729850924103,
    "time_5s": 11.731982600999217,
    "distance_5m": 12.273713131830242,
    "distance_10m": 11.389612695757071
  }
}
Does this result describe the OS1-64?

WARN

No. This dataset uses a different sensor (Hesai QT64 (64 beams) - DIFFERENT from OS1-64 (beam layout, FOV, range)); do not read it as an OS1-64 result.

Rule and evidence

dataset.same_sensor_as_target is false

{}

baseline · laptop-4c · oxford_spires

Did the pipeline meet the 100 ms deadline?

FAIL

No. 100.0 % of 888 frames missed 100 ms. Median latency 438 ms is 4.4x the deadline; P99 is 1068 ms.

Rule and evidence

pass: 0 violations; warn: 0 < violations <= 5 %; fail: violations > 5 %. Violation = per-frame pipeline latency (preprocess + register) strictly greater than the deadline.

{
  "deadline_ms": 100,
  "p50_ms": 437.8044,
  "p95_ms": 756.6902749999999,
  "p99_ms": 1067.862499,
  "max_ms": 1720.3739,
  "violation_percent": 100,
  "frames_measured": 888
}
Did registration produce valid matches?

PASS

299 of 299 registrations passed the validity gate (100.0 %).

Rule and evidence

pass: >= 99 % accepted; warn: >= 90 %; fail: < 90 %. Accepted = passes the registration validity gate.

{
  "attempts": 299,
  "accepted": 299,
  "rejected": 0,
  "rejected_by_reason": {},
  "icp_iterations_median": 10,
  "inlier_ratio_median": 0.99795,
  "residual_median_m": 0.15885
}
How accurate was the estimated trajectory?

INFO

ATE RMSE 1.32 m (max 2.79 m) over 45.6 m of ground-truth path after rigid alignment; position RPE is reported per window in the summary. Both include any unmodelled sensor-to-body offset (lever arm) between the estimate and the ground-truth body frame, so they mix drift with extrinsic bias. Scan-to-scan odometry has no loop closure, so error accumulates with distance.

Rule and evidence

descriptive; no pass/fail threshold is applied to accuracy

{
  "ate_rmse_m": 1.3217205014603846,
  "ate_max_m": 2.791091479410538,
  "gt_path_length_m": 45.563618073800534,
  "ate_rmse_percent_of_path": 2.900824292134924,
  "final_error_m": 1.2982540736844619,
  "rpe_mode": "world_position_after_umeyama_alignment",
  "rpe": {
    "time_1s": 15.048729850924103,
    "time_5s": 11.731982600999217,
    "distance_5m": 12.273713131830242,
    "distance_10m": 11.389612695757071
  }
}
Does this result describe the OS1-64?

WARN

No. This dataset uses a different sensor (Hesai QT64 (64 beams) - DIFFERENT from OS1-64 (beam layout, FOV, range)); do not read it as an OS1-64 result.

Rule and evidence

dataset.same_sensor_as_target is false

{}

baseline · laptop-1c · oxford_spires

Did the pipeline meet the 100 ms deadline?

FAIL

No. 100.0 % of 888 frames missed 100 ms. Median latency 779 ms is 7.8x the deadline; P99 is 2936 ms.

Rule and evidence

pass: 0 violations; warn: 0 < violations <= 5 %; fail: violations > 5 %. Violation = per-frame pipeline latency (preprocess + register) strictly greater than the deadline.

{
  "deadline_ms": 100,
  "p50_ms": 779.4844,
  "p95_ms": 2103.4184499999997,
  "p99_ms": 2936.4477319999996,
  "max_ms": 4978.1011,
  "violation_percent": 100,
  "frames_measured": 888
}
Did registration produce valid matches?

PASS

299 of 299 registrations passed the validity gate (100.0 %).

Rule and evidence

pass: >= 99 % accepted; warn: >= 90 %; fail: < 90 %. Accepted = passes the registration validity gate.

{
  "attempts": 299,
  "accepted": 299,
  "rejected": 0,
  "rejected_by_reason": {},
  "icp_iterations_median": 10,
  "inlier_ratio_median": 0.99795,
  "residual_median_m": 0.15885
}
How accurate was the estimated trajectory?

INFO

ATE RMSE 1.32 m (max 2.79 m) over 45.6 m of ground-truth path after rigid alignment; position RPE is reported per window in the summary. Both include any unmodelled sensor-to-body offset (lever arm) between the estimate and the ground-truth body frame, so they mix drift with extrinsic bias. Scan-to-scan odometry has no loop closure, so error accumulates with distance.

Rule and evidence

descriptive; no pass/fail threshold is applied to accuracy

{
  "ate_rmse_m": 1.3217205014603846,
  "ate_max_m": 2.791091479410538,
  "gt_path_length_m": 45.563618073800534,
  "ate_rmse_percent_of_path": 2.900824292134924,
  "final_error_m": 1.2982540736844619,
  "rpe_mode": "world_position_after_umeyama_alignment",
  "rpe": {
    "time_1s": 15.048729850924103,
    "time_5s": 11.731982600999217,
    "distance_5m": 12.273713131830242,
    "distance_10m": 11.389612695757071
  }
}
Does this result describe the OS1-64?

WARN

No. This dataset uses a different sensor (Hesai QT64 (64 beams) - DIFFERENT from OS1-64 (beam layout, FOV, range)); do not read it as an OS1-64 result.

Rule and evidence

dataset.same_sensor_as_target is false

{}

p2plane · laptop-full · oxford_spires

Did the pipeline meet the 100 ms deadline?

FAIL

No. 100.0 % of 888 frames missed 100 ms. Median latency 518 ms is 5.2x the deadline; P99 is 794 ms.

Rule and evidence

pass: 0 violations; warn: 0 < violations <= 5 %; fail: violations > 5 %. Violation = per-frame pipeline latency (preprocess + register) strictly greater than the deadline.

{
  "deadline_ms": 100,
  "p50_ms": 517.58795,
  "p95_ms": 657.842765,
  "p99_ms": 794.00649,
  "max_ms": 2876.7601,
  "violation_percent": 100,
  "frames_measured": 888
}
Did registration produce valid matches?

PASS

299 of 299 registrations passed the validity gate (100.0 %).

Rule and evidence

pass: >= 99 % accepted; warn: >= 90 %; fail: < 90 %. Accepted = passes the registration validity gate.

{
  "attempts": 299,
  "accepted": 299,
  "rejected": 0,
  "rejected_by_reason": {},
  "icp_iterations_median": 6,
  "inlier_ratio_median": 0.99794,
  "residual_median_m": 0.16729
}
How accurate was the estimated trajectory?

INFO

ATE RMSE 0.15 m (max 0.42 m) over 45.6 m of ground-truth path after rigid alignment; position RPE is reported per window in the summary. Both include any unmodelled sensor-to-body offset (lever arm) between the estimate and the ground-truth body frame, so they mix drift with extrinsic bias. Scan-to-scan odometry has no loop closure, so error accumulates with distance.

Rule and evidence

descriptive; no pass/fail threshold is applied to accuracy

{
  "ate_rmse_m": 0.15251217145307253,
  "ate_max_m": 0.4168300997306193,
  "gt_path_length_m": 45.563618073800534,
  "ate_rmse_percent_of_path": 0.3347235753009007,
  "final_error_m": 0.15346740111930274,
  "rpe_mode": "world_position_after_umeyama_alignment",
  "rpe": {
    "time_1s": 3.924105717352643,
    "time_5s": 1.5048470621703773,
    "distance_5m": 1.7766911665036935,
    "distance_10m": 1.3573782967742307
  }
}
Does this result describe the OS1-64?

WARN

No. This dataset uses a different sensor (Hesai QT64 (64 beams) - DIFFERENT from OS1-64 (beam layout, FOV, range)); do not read it as an OS1-64 result.

Rule and evidence

dataset.same_sensor_as_target is false

{}

optimized · laptop-full · newer_college_bag9

Did the pipeline meet the 100 ms deadline?

FAIL

No. 97.2 % of 798 frames missed 100 ms. Median latency 324 ms is 3.2x the deadline; P99 is 1521 ms.

Rule and evidence

pass: 0 violations; warn: 0 < violations <= 5 %; fail: violations > 5 %. Violation = per-frame pipeline latency (preprocess + register) strictly greater than the deadline.

{
  "deadline_ms": 100,
  "p50_ms": 323.54755,
  "p95_ms": 1017.8753149999999,
  "p99_ms": 1520.7788489999996,
  "max_ms": 3190.3493,
  "violation_percent": 97.24310776942356,
  "frames_measured": 798
}
Did registration produce valid matches?

PASS

269 of 269 registrations passed the validity gate (100.0 %).

Rule and evidence

pass: >= 99 % accepted; warn: >= 90 %; fail: < 90 %. Accepted = passes the registration validity gate.

{
  "attempts": 269,
  "accepted": 269,
  "rejected": 0,
  "rejected_by_reason": {},
  "icp_iterations_median": 10,
  "inlier_ratio_median": 0.99936,
  "residual_median_m": 0.12463
}
How accurate was the estimated trajectory?

INFO

ATE RMSE 0.51 m (max 1.06 m) over 22.5 m of ground-truth path after rigid alignment; position RPE is reported per window in the summary. Both include any unmodelled sensor-to-body offset (lever arm) between the estimate and the ground-truth body frame, so they mix drift with extrinsic bias. Scan-to-scan odometry has no loop closure, so error accumulates with distance.

Rule and evidence

descriptive; no pass/fail threshold is applied to accuracy

{
  "ate_rmse_m": 0.5068686215458409,
  "ate_max_m": 1.0565121346857196,
  "gt_path_length_m": 22.472984119767467,
  "ate_rmse_percent_of_path": 2.2554575700518296,
  "final_error_m": 0.19201671534112752,
  "rpe_mode": "world_position_after_umeyama_alignment",
  "rpe": {
    "time_1s": 28.56793327591613,
    "time_5s": 14.567642505063228,
    "distance_5m": 10.553605423564546,
    "distance_10m": 7.834970613283673
  }
}

optimized · laptop-4c · newer_college_bag9

Did the pipeline meet the 100 ms deadline?

FAIL

No. 99.9 % of 798 frames missed 100 ms. Median latency 611 ms is 6.1x the deadline; P99 is 2694 ms.

Rule and evidence

pass: 0 violations; warn: 0 < violations <= 5 %; fail: violations > 5 %. Violation = per-frame pipeline latency (preprocess + register) strictly greater than the deadline.

{
  "deadline_ms": 100,
  "p50_ms": 611.3552999999999,
  "p95_ms": 1746.0251549999998,
  "p99_ms": 2693.608919,
  "max_ms": 6835.5067,
  "violation_percent": 99.87468671679198,
  "frames_measured": 798
}
Did registration produce valid matches?

PASS

269 of 269 registrations passed the validity gate (100.0 %).

Rule and evidence

pass: >= 99 % accepted; warn: >= 90 %; fail: < 90 %. Accepted = passes the registration validity gate.

{
  "attempts": 269,
  "accepted": 269,
  "rejected": 0,
  "rejected_by_reason": {},
  "icp_iterations_median": 10,
  "inlier_ratio_median": 0.99936,
  "residual_median_m": 0.12463
}
How accurate was the estimated trajectory?

INFO

ATE RMSE 0.51 m (max 1.06 m) over 22.5 m of ground-truth path after rigid alignment; position RPE is reported per window in the summary. Both include any unmodelled sensor-to-body offset (lever arm) between the estimate and the ground-truth body frame, so they mix drift with extrinsic bias. Scan-to-scan odometry has no loop closure, so error accumulates with distance.

Rule and evidence

descriptive; no pass/fail threshold is applied to accuracy

{
  "ate_rmse_m": 0.5068686215458409,
  "ate_max_m": 1.0565121346857196,
  "gt_path_length_m": 22.472984119767467,
  "ate_rmse_percent_of_path": 2.2554575700518296,
  "final_error_m": 0.19201671534112752,
  "rpe_mode": "world_position_after_umeyama_alignment",
  "rpe": {
    "time_1s": 28.56793327591613,
    "time_5s": 14.567642505063228,
    "distance_5m": 10.553605423564546,
    "distance_10m": 7.834970613283673
  }
}

optimized · laptop-1c · newer_college_bag9

Did the pipeline meet the 100 ms deadline?

FAIL

No. 99.9 % of 798 frames missed 100 ms. Median latency 870 ms is 8.7x the deadline; P99 is 2808 ms.

Rule and evidence

pass: 0 violations; warn: 0 < violations <= 5 %; fail: violations > 5 %. Violation = per-frame pipeline latency (preprocess + register) strictly greater than the deadline.

{
  "deadline_ms": 100,
  "p50_ms": 869.7582,
  "p95_ms": 2090.321995,
  "p99_ms": 2808.401579999999,
  "max_ms": 4922.3288,
  "violation_percent": 99.87468671679198,
  "frames_measured": 798
}
Did registration produce valid matches?

PASS

269 of 269 registrations passed the validity gate (100.0 %).

Rule and evidence

pass: >= 99 % accepted; warn: >= 90 %; fail: < 90 %. Accepted = passes the registration validity gate.

{
  "attempts": 269,
  "accepted": 269,
  "rejected": 0,
  "rejected_by_reason": {},
  "icp_iterations_median": 10,
  "inlier_ratio_median": 0.99936,
  "residual_median_m": 0.12463
}
How accurate was the estimated trajectory?

INFO

ATE RMSE 0.51 m (max 1.06 m) over 22.5 m of ground-truth path after rigid alignment; position RPE is reported per window in the summary. Both include any unmodelled sensor-to-body offset (lever arm) between the estimate and the ground-truth body frame, so they mix drift with extrinsic bias. Scan-to-scan odometry has no loop closure, so error accumulates with distance.

Rule and evidence

descriptive; no pass/fail threshold is applied to accuracy

{
  "ate_rmse_m": 0.5068686215458409,
  "ate_max_m": 1.0565121346857196,
  "gt_path_length_m": 22.472984119767467,
  "ate_rmse_percent_of_path": 2.2554575700518296,
  "final_error_m": 0.19201671534112752,
  "rpe_mode": "world_position_after_umeyama_alignment",
  "rpe": {
    "time_1s": 28.56793327591613,
    "time_5s": 14.567642505063228,
    "distance_5m": 10.553605423564546,
    "distance_10m": 7.834970613283673
  }
}

optimized · laptop-full · oxford_spires

Did the pipeline meet the 100 ms deadline?

FAIL

No. 98.8 % of 888 frames missed 100 ms. Median latency 248 ms is 2.5x the deadline; P99 is 1056 ms.

Rule and evidence

pass: 0 violations; warn: 0 < violations <= 5 %; fail: violations > 5 %. Violation = per-frame pipeline latency (preprocess + register) strictly greater than the deadline.

{
  "deadline_ms": 100,
  "p50_ms": 247.53379999999999,
  "p95_ms": 456.6521599999999,
  "p99_ms": 1055.5644659999998,
  "max_ms": 3694.6012,
  "violation_percent": 98.76126126126125,
  "frames_measured": 888
}
Did registration produce valid matches?

PASS

299 of 299 registrations passed the validity gate (100.0 %).

Rule and evidence

pass: >= 99 % accepted; warn: >= 90 %; fail: < 90 %. Accepted = passes the registration validity gate.

{
  "attempts": 299,
  "accepted": 299,
  "rejected": 0,
  "rejected_by_reason": {},
  "icp_iterations_median": 10,
  "inlier_ratio_median": 0.99795,
  "residual_median_m": 0.15885
}
How accurate was the estimated trajectory?

INFO

ATE RMSE 1.32 m (max 2.79 m) over 45.6 m of ground-truth path after rigid alignment; position RPE is reported per window in the summary. Both include any unmodelled sensor-to-body offset (lever arm) between the estimate and the ground-truth body frame, so they mix drift with extrinsic bias. Scan-to-scan odometry has no loop closure, so error accumulates with distance.

Rule and evidence

descriptive; no pass/fail threshold is applied to accuracy

{
  "ate_rmse_m": 1.3217205014603846,
  "ate_max_m": 2.791091479410538,
  "gt_path_length_m": 45.563618073800534,
  "ate_rmse_percent_of_path": 2.900824292134924,
  "final_error_m": 1.2982540736844619,
  "rpe_mode": "world_position_after_umeyama_alignment",
  "rpe": {
    "time_1s": 15.048729850924103,
    "time_5s": 11.731982600999217,
    "distance_5m": 12.273713131830242,
    "distance_10m": 11.389612695757071
  }
}
Does this result describe the OS1-64?

WARN

No. This dataset uses a different sensor (Hesai QT64 (64 beams) - DIFFERENT from OS1-64 (beam layout, FOV, range)); do not read it as an OS1-64 result.

Rule and evidence

dataset.same_sensor_as_target is false

{}

baseline · laptop-full · newer_college_bag9

Did the pipeline meet the 100 ms deadline?

FAIL

No. 100.0 % of 1330 frames missed 100 ms. Median latency 887 ms is 8.9x the deadline; P99 is 4124 ms.

Rule and evidence

pass: 0 violations; warn: 0 < violations <= 5 %; fail: violations > 5 %. Violation = per-frame pipeline latency (preprocess + register) strictly greater than the deadline.

{
  "deadline_ms": 100,
  "p50_ms": 887.4585,
  "p95_ms": 2069.0244699999994,
  "p99_ms": 4124.290643000001,
  "max_ms": 8970.7056,
  "violation_percent": 100,
  "frames_measured": 1330
}
Did registration produce valid matches?

PASS

269 of 269 registrations passed the validity gate (100.0 %).

Rule and evidence

pass: >= 99 % accepted; warn: >= 90 %; fail: < 90 %. Accepted = passes the registration validity gate.

{
  "attempts": 269,
  "accepted": 269,
  "rejected": 0,
  "rejected_by_reason": {},
  "icp_iterations_median": 10,
  "inlier_ratio_median": 0.99936,
  "residual_median_m": 0.12463
}
How accurate was the estimated trajectory?

INFO

ATE RMSE 0.51 m (max 1.06 m) over 22.5 m of ground-truth path after rigid alignment; position RPE is reported per window in the summary. Both include any unmodelled sensor-to-body offset (lever arm) between the estimate and the ground-truth body frame, so they mix drift with extrinsic bias. Scan-to-scan odometry has no loop closure, so error accumulates with distance.

Rule and evidence

descriptive; no pass/fail threshold is applied to accuracy

{
  "ate_rmse_m": 0.5068686215458409,
  "ate_max_m": 1.0565121346857196,
  "gt_path_length_m": 22.472984119767467,
  "ate_rmse_percent_of_path": 2.2554575700518296,
  "final_error_m": 0.19201671534112752,
  "rpe_mode": "world_position_after_umeyama_alignment",
  "rpe": {
    "time_1s": 28.56793327591613,
    "time_5s": 14.567642505063228,
    "distance_5m": 10.553605423564546,
    "distance_10m": 7.834970613283673
  }
}

optimized · laptop-full · newer_college_bag9

Did the pipeline meet the 100 ms deadline?

FAIL

No. 98.8 % of 1330 frames missed 100 ms. Median latency 506 ms is 5.1x the deadline; P99 is 3642 ms.

Rule and evidence

pass: 0 violations; warn: 0 < violations <= 5 %; fail: violations > 5 %. Violation = per-frame pipeline latency (preprocess + register) strictly greater than the deadline.

{
  "deadline_ms": 100,
  "p50_ms": 505.83180000000004,
  "p95_ms": 2301.0204099999996,
  "p99_ms": 3642.4675620000053,
  "max_ms": 10755.3086,
  "violation_percent": 98.796992481203,
  "frames_measured": 1330
}
Did registration produce valid matches?

PASS

269 of 269 registrations passed the validity gate (100.0 %).

Rule and evidence

pass: >= 99 % accepted; warn: >= 90 %; fail: < 90 %. Accepted = passes the registration validity gate.

{
  "attempts": 269,
  "accepted": 269,
  "rejected": 0,
  "rejected_by_reason": {},
  "icp_iterations_median": 10,
  "inlier_ratio_median": 0.99936,
  "residual_median_m": 0.12463
}
How accurate was the estimated trajectory?

INFO

ATE RMSE 0.51 m (max 1.06 m) over 22.5 m of ground-truth path after rigid alignment; position RPE is reported per window in the summary. Both include any unmodelled sensor-to-body offset (lever arm) between the estimate and the ground-truth body frame, so they mix drift with extrinsic bias. Scan-to-scan odometry has no loop closure, so error accumulates with distance.

Rule and evidence

descriptive; no pass/fail threshold is applied to accuracy

{
  "ate_rmse_m": 0.5068686215458409,
  "ate_max_m": 1.0565121346857196,
  "gt_path_length_m": 22.472984119767467,
  "ate_rmse_percent_of_path": 2.2554575700518296,
  "final_error_m": 0.19201671534112752,
  "rpe_mode": "world_position_after_umeyama_alignment",
  "rpe": {
    "time_1s": 28.56793327591613,
    "time_5s": 14.567642505063228,
    "distance_5m": 10.553605423564546,
    "distance_10m": 7.834970613283673
  }
}

single-frame · newer_college_sparse

What does preprocessing alone cost per frame?

FAIL

Preprocessing (invalid-point removal, range filter, voxel downsample) takes a median 143 ms (P95 186 ms); the voxel stage alone is 133 ms. That already exceeds the deadline before any registration. Non-consecutive frames: no odometry or accuracy claim.

Rule and evidence

fail if the median preprocessing latency exceeds the deadline

{
  "deadline_ms": 100,
  "preprocess_p50_ms": 142.6959,
  "preprocess_p95_ms": 186.46445,
  "voxel_p50_ms": 133.3462,
  "frames": 591
}

Metric definitions

MetricUnitEvidenceDefinitionCalculationInterpretation
Pipeline latencymsMEASUREDWall-clock time to process one frame that is already in memory: invalid-point removal, range filter, voxel downsample, ICP registration, validity gate and pose update.time.perf_counter around each stage in the worker process; total = preprocess + register. Warm-up frames and the initial reference frame are excluded.Compare with the sensor period (100 ms at 10 Hz). Single-machine software timing; not a hard real-time guarantee.
End-to-end latencymsMEASUREDPipeline latency plus the time to read the frame from disk.load_ms + total_ms.A live sensor driver delivers frames in memory, so total_ms is the primary number; e2e shows the file-I/O cost of offline replay.
P50 / P90 / P95 / P99 / maxmsDERIVEDOrder statistics of per-frame latency pooled over all repeats.numpy.percentile, linear interpolation.P99 and max describe the tail that causes missed frames.
Deadline violation rate% of framesDERIVEDShare of measured frames whose pipeline latency is strictly greater than the sensor period.count(total_ms > deadline) / frames measured.The deadline statistic. A P95 below the deadline still allows 5 % of frames to miss it.
Sustained throughputframes/sMEASUREDFrames processed per second of wall-clock time over the whole loop, file loading included.frames / loop wall time per repeat (min, median, max over repeats).Must exceed the sensor rate (10 fps) to keep up in steady state.
Peak memoryMBMEASUREDPeak resident (working-set) memory of the worker process.psutil peak_wset (Windows) or ru_maxrss.Process peak; machine RAM is a separate specification, not a measurement.
Accepted registrations%DERIVEDRegistrations that pass the validity gate (inlier ratio, residual, plausible motion) divided by attempts.Frame 0 is excluded (it has no registration). Thresholds are in run.json.Rejections are counted by reason, never silently accepted.
Absolute trajectory error (RMSE)mDERIVEDRMS position error between the estimated and ground-truth trajectories after a rigid SE(3) Umeyama alignment (no scale), using timestamp-associated poses.Association within 20 ms, one-to-one.Needs ground truth. Without loop closure error grows with distance. It also includes any unmodelled sensor-to-body offset (lever arm) between the estimated sensor positions and the ground-truth body positions, so it mixes drift with extrinsic bias.
Relative pose error (position)m, % of segmentDERIVEDError of the displacement over fixed time or path-length windows, in the aligned world frame.Rotation RPE is UNAVAILABLE because the ground-truth body frame differs from the sensor frame (DEC-012). Position RPE removes the step-direction (yaw) offset but not a lever arm, and the global alignment is fitted over the whole window, so it is mildly optimistic.Drift per unit motion; less sensitive to a single bad alignment than ATE.
ICP iterationscountMEASUREDIterations before the convergence rule stopped (cap 30).Per registration.Drives register_ms; hitting the cap flags poor initial alignment.
Inlier ratiofractionMEASUREDSource points with a target neighbour within 1.5 m, divided by source points.Last ICP iteration.Low values mean little scan overlap or a wrong match.
Registration residualmMEASUREDMean distance of inlier correspondences at the start of the last iteration.Point-to-point distance for all variants.Included in the validity gate.