Benchmarks

Results of detection models on the validation split.

Method Vehicle Pedestrian Cyclist mAP
overall 0-30m 30-50m 50m-inf overall 0-30m 30-50m 50m-inf overall 0-30m 30-50m 50m-inf
PointRCNN 52.09 74.45 40.89 16.81 4.28 6.17 2.40 0.91 29.84 46.03 20.94 5.46 28.74
PointPillars 68.57 80.86 62.07 47.04 17.63 19.74 15.15 10.23 46.81 58.33 40.32 25.86 44.34
SECOND 71.19 84.04 63.02 47.25 26.44 29.33 24.05 18.05 58.04 69.96 52.43 34.61 51.89
PV-RCNN 77.77 89.39 72.55 58.64 23.50 25.61 22.84 17.27 59.37 71.66 52.58 36.17 53.55
CenterPoints 66.79 80.10 59.55 43.39 49.90 56.24 42.61 26.27 63.45 74.28 57.94 41.48 60.05
PointPainting 66.17 80.31 59.80 42.26 44.84 52.63 36.63 22.47 62.34 73.55 57.20 40.39 57.78

Results of detection models on the testing split.

Method Vehicle Pedestrian Cyclist mAP
overall 0-30m 30-50m 50m-inf overall 0-30m 30-50m 50m-inf overall 0-30m 30-50m 50m-inf
PointRCNN 52.00 74.44 40.72 22.14 8.73 12.20 6.96 2.96 34.02 46.48 27.39 11.45 31.58
PointPillars 69.52 84.51 60.55 45.72 17.28 20.21 15.06 11.48 49.63 60.15 42.43 27.73 45.47
SECOND 69.71 86.96 60.22 43.02 26.09 30.52 24.63 14.19 59.92 70.54 54.89 34.34 51.90
PV-RCNN 76.98 89.89 69.35 55.52 22.66 27.23 21.28 12.08 61.93 72.13 56.64 37.23 53.85
CenterPoints 66.35 83.65 56.74 41.57 51.80 62.80 45.41 24.53 65.57 73.02 62.85 44.77 61.24
PointPainting 66.46 83.70 56.89 40.74 47.62 58.95 39.33 23.34 65.27 73.48 61.53 43.90 59.78

Results of self-supervised learning methods on the validation split.

Method Vehicle Pedestrian Cyclist mAP
overall 0-30m 30-50m 50m-inf overall 0-30m 30-50m 50m-inf overall 0-30m 30-50m 50m-inf
baseline (SECOND) 71.19 84.04 63.02 47.25 26.44 29.33 24.05 18.05 58.04 69.96 52.43 34.61 51.89
Small
BYOL 68.02 81.01 60.21 44.17 19.50 22.16 16.68 12.06 50.61 62.46 44.29 28.18 46.04
PointContrast 71.07 83.31 64.90 49.34 22.52 23.73 21.81 16.06 56.36 68.11 50.35 34.06 49.98
SwAV 72.71 83.68 65.91 50.10 25.13 27.77 22.77 16.36 58.05 69.99 52.23 34.86 51.96
DeepCluster 73.19 84.25 66.86 50.47 24.00 26.36 21.73 16.79 58.99 70.80 53.66 36.17 52.06
DepthContrast 71.88 84.26 65.58 49.97 23.57 26.36 21.15 14.39 56.63 68.26 50.82 34.67 50.69
Medium
BYOL 70.93 84.15 63.48 45.74 25.86 29.91 21.55 15.83 55.63 58.59 49.01 29.53 50.82
PointContrast 71.39 83.89 65.22 47.73 27.69 32.53 23.00 14.68 56.88 69.01 50.41 34.57 51.99
SwAV 72.51 83.39 65.46 51.08 27.08 29.94 25.19 17.13 57.85 69.87 52.38 33.78 52.48
DeepCluster 71.62 83.99 65.55 50.77 29.33 33.25 25.08 17.00 57.61 68.57 52.58 34.05 52.86
DepthContrast 71.92 84.38 65.86 48.48 29.01 33.09 24.23 15.88 57.51 69.86 51.00 34.41 52.81
Large
BYOL 71.32 83.59 64.89 50.27 25.02 27.06 22.96 17.04 58.56 70.18 52.74 36.32 51.63
PointContrast 71.87 86.93 62.85 48.65 28.03 33.07 25.91 14.44 60.88 71.12 55.77 36.78 53.59
SwAV 72.46 83.09 66.66 51.50 29.84 34.15 26.22 17.61 57.84 68.79 52.21 35.39 53.38
DeepCluster 72.89 83.52 67.09 50.38 30.32 34.76 26.43 18.33 57.94 69.18 52.42 34.36 53.72

Results of self-supervised learning methods on the testing split.

Method Vehicle Pedestrian Cyclist mAP
overall 0-30m 30-50m 50m-inf overall 0-30m 30-50m 50m-inf overall 0-30m 30-50m 50m-inf
baseline (SECOND) 69.71 86.96 60.22 43.02 26.09 30.52 24.63 14.19 59.92 70.54 54.89 34.34 51.9
Small
BYOL 67.57 84.61 58.26 41.59 17.22 19.45 16.71 10.43 53.36 64.95 47.47 27.66 46.05
PointContrast 71.53 87.02 62.37 47.23 22.68 26.33 21.58 12.98 58.04 70.01 51.74 31.69 50.75
SwAV 72.25 87.20 63.38 48.93 25.11 29.32 23.50 14.13 60.67 70.90 55.91 35.39 52.68
DeepCluster 72.06 87.09 63.09 48.78 27.56 32.21 26.60 13.61 50.30 70.33 55.82 35.89 53.31
DepthContrast 72.06 87.33 62.89 47.99 23.69 27.93 22.29 11.80 58.66 70.46 52.81 33.17 51.47
Medium
BYOL 69.69 84.83 60.41 46.05 27.31 32.58 24.60 13.69 57.22 69.57 51.07 29.15 51.41
PointContrast 70.15 86.71 61.12 48.11 29.23 35.52 36.28 13.06 58.91 70.05 53.86 34.27 52.76
SwAV 72.10 87.11 63.15 48.58 28.00 33.10 25.88 14.19 60.17 70.46 55.61 34.84 53.42
DeepCluster 72.12 87.31 62.97 48.55 30.06 36.07 27.23 13.47 60.45 70.81 54.93 36.03 54.21
DepthContrast 71.65 86.96 61.52 48.74 30.46 36.82 27.14 15.13 58.83 69.90 54.06 33.99 53.64
Large
BYOL 72.23 87.30 63.13 48.31 23.62 27.10 22.14 13.47 60.45 70.82 55.31 35.65 52.10
PointContrast 73.15 83.92 67.29 50.97 27.48 31.45 24.17 16.70 58.33 70.37 52.26 35.61 52.99
SwAV 71.96 86.92 62.83 48.85 30.60 36.42 28.03 14.52 60.27 70.43 55.52 36.25 54.28
DeepCluster 71.85 86.96 62.91 48.54 30.54 37.08 27.55 13.86 60.42 70.60 55.47 36.29 54.27

Results of semi-supervised learning methods on the validation split.

Method Vehicle Pedestrian Cyclist mAP
overall 0-30m 30-50m 50m-inf overall 0-30m 30-50m 50m-inf overall 0-30m 30-50m 50m-inf
baseline (SECOND) 71.19 84.04 63.02 47.25 26.44 29.33 24.05 18.05 58.04 69.96 52.43 34.61 51.89
Small
Pseudo Label 72.80 84.46 64.97 51.46 25.50 28.36 22.66 18.51 55.37 65.95 50.34 34.42 51.22
Noisy Student 73.69 84.69 67.72 53.41 28.81 33.23 23.42 16.93 54.67 65.58 50.43 32.65 52.39
Mean Teacher 74.46 86.65 68.44 53.59 30.54 34.24 26.31 20.12 61.02 72.51 55.24 39.11 55.34
SESS 73.33 84.52 66.22 52.83 27.31 31.11 23.94 19.01 59.52 71.03 53.93 36.68 53.39
3DIoUMatch 73.81 84.61 68.11 54.48 30.86 35.87 25.55 18.30 56.77 68.02 51.80 35.91 53.81
Medium
Pseudo Label 73.03 86.06 65.96 51.42 24.56 27.28 20.81 17.00 53.61 65.26 48.44 33.58 50.40
Noisy Student 75.53 86.52 69.78 55.05 31.56 35.80 26.24 21.21 58.93 69.61 53.73 36.94 55.34
Mean Teacher 76.01 86.47 70.34 55.92 35.58 40.86 30.44 19.82 63.21 74.89 56.77 40.29 58.27
SESS 72.11 84.06 66.44 53.61 33.44 38.58 28.10 18.67 61.82 73.20 56.60 38.73 55.79
3DIoUMatch 75.69 86.46 70.22 56.06 34.14 38.84 29.19 19.62 58.93 69.08 54.16 38.87 56.25
Large
Pseudo Label 72.41 84.06 64.54 50.05 23.62 26.80 20.13 16.66 53.25 64.69 48.52 33.47 49.76
Noisy Student 75.99 86.67 70.48 55.60 33.31 37.81 28.19 21.39 59.81 70.01 55.13 38.33 56.37
Mean Teacher 76.38 86.45 70.99 57.48 35.95 41.76 29.05 18.81 65.50 75.72 60.07 43.66 59.28
SESS 75.95 86.83 70.45 55.76 34.43 40.00 27.92 19.20 63.58 74.85 58.88 39.51 57.99
3DIoUMatch 75.81 86.11 71.82 57.84 35.70 40.68 30.34 21.15 59.69 70.69 54.92 39.08 57.07

Results of semi-supervised learning methods on the testing split.

Method Vehicle Pedestrian Cyclist mAP
overall 0-30m 30-50m 50m-inf overall 0-30m 30-50m 50m-inf overall 0-30m 30-50m 50m-inf
baseline (SECOND) 69.71 86.96 60.22 43.02 26.09 30.52 24.63 14.19 59.92 70.54 54.89 34.34 51.9
Small
Pseudo Label 71.05 86.51 61.81 47.49 25.58 31.03 22.03 14.12 58.08 68.5 52.63 35.61 51.57
Noisy Student 73.25 88.84 64.61 49.95 28.04 34.62 23.43 14.2 57.58 67.77 53.43 33.76 52.96
Mean Teacher 74.13 89.34 65.28 50.91 31.66 37.44 29.9 14.61 62.69 71.88 59.22 39.45 56.16
SESS 72.42 87.23 63.55 49.11 27.32 32.26 24.47 15.36 61.76 72.39 57.29 37.33 53.83
3DIoUMatch 72.12 87.05 63.65 50.35 31.41 38.56 27.62 14.25 59.46 69.53 54.82 36.18 54.33
Medium
Pseudo Label 70.72 86.21 61.72 47.39 21.74 25.73 19.91 13.28 56.01 67.14 50.18 33.23 49.49
Noisy Student 73.97 89.09 65.35 51.04 30.32 36.24 27.08 16.24 61.35 71.28 56.7 37.96 55.22
Mean Teacher 74.71 89.28 66.1 52.9 36.03 42.97 33.29 18.7 64.88 74.05 60.8 42.63 58.54
SESS 72.6 87.02 64.29 50.68 35.23 42.59 31.4 16.64 64.67 73.93 61.14 40.8 57.5
3DIoUMatch 74.26 89.08 66.11 53.03 33.91 41.02 30.07 16.15 61.3 71.29 56.49 38.13 56.49
Large
Pseudo Label 70.29 85.94 61.18 46.66 21.85 25.83 20.22 12.75 55.72 66.96 50.29 32.92 49.29
Noisy Student 74.5 89.23 67.11 53.15 33.28 40.2 28.89 17.5 62.05 71.76 57.53 39.32 56.61
Mean Teacher 76.6 89.41 68.29 55.66 36.37 43.84 32.49 17.11 66.99 75.87 63.35 44.06 59.99
SESS 74.52 88.97 66.32 52.47 36.29 43.53 33.15 16.68 65.52 74.63 62.67 41.91 58.78
3DIoUMatch 74.48 89.13 66.35 54.59 35.74 43.35 32.08 17.34 62.06 71.86 58.00 39.09 57.43

Results on unsupervised domain adaptation.

Task Waymo->ONCE nuScenes->ONCE ONCE->KITTI
Method AP_BEV AP_3D AP_BEV AP_3D AP_BEV AP_3D
Source Only 65.55 32.88 46.85 23.74 42.01 12.11
SN 67.97 38.25 62.47 29.53 48.12 21.12
ST3D 68.05 48.34 42.53 17.52 86.89 41.42
Oracle 89.00 77.50 89.00 77.50 83.29 73.45