ModelsVideos
Five models × six videos
Five models on six hard videos (man_airport, man_eyeline, man_park, man_winter, man_standing_rotate, breakdance (frames 590-1180)). Each video (for breakdance: frames 590-1180 of the video) is run in a single forward pass (centre crop to 16:9, 672×378); for every model the page shows depth and depth-only (oblique 30°) videos, an interactive 3D point cloud and temporal-consistency metrics.
Models
- PRETRAINED_PI3X: the original pretrained Pi3X, no fine-tuning.
- V1_PLAIN_FINETUNE: Pi3X fine-tuned on our self-collected dataset, with no other modification.
- V2_3D_MOTION_LOSS: V1_PLAIN_FINETUNE plus a 3D motion loss.
- V3_2D_REPROJ_LOSS: V2_3D_MOTION_LOSS plus a 2D reprojection loss.
- V4_2ND_DEV_CAM_LOSS: V3_2D_REPROJ_LOSS plus a second-derivative (acceleration) loss on the camera.
Video classes
Class rules, measurements and thresholds
- Camera motion (measured): the optical-flow magnitude of background pixels, mean over the video, in % of the frame width per second. static < 1 ≤ small < 5 ≤ medium < 18 ≤ large; "static" also needs the background to travel less than 3 % of the frame width in total (the farthest it moves on screen), otherwise "small", to tell a locked-off camera from a slow drift. Flow rather than predicted poses, otherwise the poses being evaluated would classify the videos; a mostly distant background gives low flow; less reliable with little or textureless background.
- Object motion (measured): the flow of the object pixels minus a background affine model, mean over the video, same unit. small < 6 ≤ medium < 15 ≤ large < 30 ≤ very large. This is motion on screen: motion along the line of sight hardly shows; with a translating camera, objects near the camera get parallax and higher values.
- Object screen area (measured): mean share of the frame covered by the SegFormer dynamic-object mask. small < 14 % ≤ medium < 45 % ≤ large. The mask includes people in the background.
- Object size (real) (by eye): the size of the object itself, regardless of framing: small < 1 m, medium 1–2 m (a person), large > 2 m (horse, vehicle). The metric depth of the models is not usable as a size on these videos.
- Object count, kind (by eye), from the mask contact sheets.
All videos
| Video | Camera motion | Object motion | Object size (real) | Object screen area | Object count | Object kind | Length (s) | |
|---|---|---|---|---|---|---|---|---|
| man_airport | small3.1 % width/s, 74 % width in total | medium11.9 % width/s | mediumone person in front, a crowd walking in the background | medium41 % | crowd | person | 26.5 | |
| man_eyeline | static0.7 % width/s, 2 % width in total | small4.0 % width/s | mediumstudio with blue / red stage lights and EYELINE screens: a man talks to the camera and gestures; one or two crew members far behind; the camera moves slowly | medium19 % | single | person | 6.5 | |
| man_park | medium7.4 % width/s, 145 % width in total | medium8.8 % width/s | mediumlakeside park path: a man talks while walking towards the camera, which tracks backwards; pedestrians, a stroller and a dog pass behind him, a group of pedestrians far behind later | medium21 % | crowd | person + animal | 19.8 | |
| man_winter | medium5.3 % width/s, 73 % width in total | large15.5 % width/s | mediumsnowy park at sunset: a man talks while walking towards the backward-moving camera; people in coats pass very close in front (large transient occluders); children sledding and pedestrians behind | medium38 % | crowd | person | 14.1 | |
| man_standing_rotate | unclassified– | unclassified– | unclassified– | unclassified– | unclassified– | unclassified– | 11.0 | |
| breakdance (frames 590-1180) | medium8.0 % width/s, 133 % width in total | large20.7 % width/s | mediuma breakdancer (frames 590-1180 of the video) | small11 % | single | person | 24.6 |
How to read the videos, the 3D point cloud and the metrics
How to read this page
- Input: every frame of each video at its native fps, centre-cropped to 16:9 and resized to 672×378; the whole video in one forward pass (long videos use a chunked but mathematically equivalent forward).
- depth: inverse depth, turbo colour map; the colour range is the 2–98 % range of the whole video, fixed over the video, so colour flicker = flicker of the predicted depth.
- confidence (hidden by default; tick "confidence" above): sigmoid(conf), viridis 0–1. The point clouds draw only points with conf > 0.1 that are not on depth edges (the rule of the Pi3 example script).
- Dynamic objects: SegFormer-B0 (ADE20K, the weights shipped with Pi3) marks people, animals and vehicles as dynamic objects (red); everything else is static background, and the per-pixel jitter is measured separately for the two groups. Segmentation errors (e.g. a missed person) leak into "static": compare with the mask video.
- Depth only (30° oblique) (a collapsed section on every card): the points of the current frame stay in their own camera coordinates (no pose) and are seen from 30° to the left of the subject; the virtual camera turns about the median subject depth of the video and stays fixed. Pose errors do not appear at all; what still moves is depth jumping back and forth along the line of sight (seen from the side as a sideways shift of about sin 30° = half the depth error). Real forward / backward motion of the subject also becomes a sideways shift, and the subject may leave the view when it is much nearer or farther than the median; use the 3D viewer then.
- Interactive 3D point cloud (bottom of each card, one open at a time): about 16,000 points per frame (one per 4×4 pixels), by default only points with conf ≥ 0.1 that are not on depth edges, in a fixed world view: every model's points are placed in its own first-frame camera coordinates (that camera at the origin, looking along +z), and the models are stacked top to bottom with a shared time. With "link views" (on by default) all panels share one virtual camera: the same viewing direction and field of view (the median of the five predicted focal lengths), and each model's cloud scaled by the median depth of the static background pixels (outside the SegFormer objects) that all five models keep in that first frame, so that the static scenes of the five models coincide; what still differs between the panels is the models' own geometry and camera motion. The optional camera frusta and image planes use each model's own predicted focal length. Accumulating 4–8 frames with the time colour shows whether the static background stays one thin layer; "accumulate static background only" keeps the moving objects to the current frame. The model boxes inside the viewer (also in full screen) and "Models" above show or hide each model.
- Metrics (proxies, no ground truth): mean second difference (jitter) of translation, rotation and the per-frame mean log depth; speed = mean first difference; jitter / speed is near 0 for smooth motion and about 1.7 for pure noise. Per-pixel jitter: WAFT optical flow (WAFT-DAv2-a2) chains pixels into three-frame tracks t−1, t, t+1, keeping only pixels that are forward-backward consistent (≤ 1 px), textured (a Shi-Tomasi threshold: WAFT underestimates motion on textureless white walls) and valid points in all three frames; depth jitter = |Δ² log z| (camera coordinates, no pose), world jitter = |Δ² X| ÷ z (with pose); per-frame median, then mean over the video, in %. Region depth jitter = the absolute value of the median of the signed Δ² log z over the same tracks, i.e. the part where a whole region jumps back and forth together (the object as a whole moving nearer / farther); it is never larger than the per-pixel depth jitter. The two close together → the error is global and a per-frame scale / shift correction (two parameters) can handle it; region much smaller than per-pixel → the error is a local change of shape (e.g. a deforming hand) that needs per-point constraints. Computed along tracks, so it is not affected by the frame-to-frame flicker of the mask edges. Focal jitter = |Δ² log f| of the focal length implied by each frame's predicted rays (least squares of (u, v) = f·(x/z, y/z), principal point at the image centre): Pi3X predicts the rays anew in every frame, and focal jumps make the points zoom in and out relative to the image. The card also lists the median focal length. At 24–30 fps real motion is smooth, so the second difference of the static background is almost all prediction jitter (its true value in world coordinates is 0). The dynamic-object values include real acceleration: fast parts close to the camera have a real Δ² that is not small, so compare the dynamic group only between models on the same video, not across videos or with the static group. A frame's group is skipped with fewer than 100 valid tracks (of a 95×168 grid); "–" when fewer than 25 % of the frames have a value. SegFormer works by class, so parked cars or people in murals count as dynamic too; those pixels only make the static group smaller, they do not leak into it. The values are for comparing models, not an absolute quality.
No video matches the filters.
man_airport
Camera smallMotion mediumSize mediumArea mediumCount crowdKind person
source 784×440 · 24.00 fps · 636 frames · 26.5 s
PRETRAINED_PI3X: inference 753 s (peak 20 GB) · metric scale 0.31 · geometric-mean depth 4.09 · subject depth 2.26 · translation 0.32 / jitter 0.47 % · rotation 0.08 / jitter 0.08° · focal 817 px / jitter 0.25 % · whole-frame depth jitter 1.59 % · region depth jitter static / dyn. 0.19 / 0.25 % · per-pixel depth jitter static / dyn. 0.33 / 0.47 % · per-pixel world jitter static / dyn. 0.41 / 1.36 % (frames with enough tracks: static 100% / dyn. 100%)
V1_PLAIN_FINETUNE: inference 192 s (peak 21 GB) · metric scale 0.29 · geometric-mean depth 6.64 · subject depth 4.07 · translation 0.34 / jitter 0.52 % · rotation 0.08 / jitter 0.08° · focal 852 px / jitter 0.28 % · whole-frame depth jitter 1.41 % · region depth jitter static / dyn. 0.23 / 0.28 % · per-pixel depth jitter static / dyn. 0.38 / 0.51 % · per-pixel world jitter static / dyn. 0.47 / 1.35 % (frames with enough tracks: static 100% / dyn. 100%)
V2_3D_MOTION_LOSS: inference 193 s (peak 21 GB) · metric scale 0.29 · geometric-mean depth 6.92 · subject depth 4.26 · translation 0.26 / jitter 0.39 % · rotation 0.07 / jitter 0.06° · focal 870 px / jitter 0.22 % · whole-frame depth jitter 1.31 % · region depth jitter static / dyn. 0.18 / 0.24 % · per-pixel depth jitter static / dyn. 0.32 / 0.46 % · per-pixel world jitter static / dyn. 0.36 / 1.03 % (frames with enough tracks: static 100% / dyn. 100%)
V3_2D_REPROJ_LOSS: inference 188 s (peak 21 GB) · metric scale 0.29 · geometric-mean depth 6.94 · subject depth 4.25 · translation 0.23 / jitter 0.34 % · rotation 0.07 / jitter 0.06° · focal 848 px / jitter 0.20 % · whole-frame depth jitter 1.25 % · region depth jitter static / dyn. 0.17 / 0.24 % · per-pixel depth jitter static / dyn. 0.32 / 0.46 % · per-pixel world jitter static / dyn. 0.35 / 0.90 % (frames with enough tracks: static 100% / dyn. 100%)
V4_2ND_DEV_CAM_LOSS: inference 749 s (peak 20 GB) · metric scale 0.29 · geometric-mean depth 6.82 · subject depth 4.17 · translation 0.17 / jitter 0.22 % · rotation 0.07 / jitter 0.04° · focal 850 px / jitter 0.14 % · whole-frame depth jitter 1.14 % · region depth jitter static / dyn. 0.13 / 0.21 % · per-pixel depth jitter static / dyn. 0.29 / 0.43 % · per-pixel world jitter static / dyn. 0.31 / 0.69 % (frames with enough tracks: static 100% / dyn. 100%)
(controls all videos of this card)
Depth
Depth only (own camera coordinates, fixed oblique 30° view, no pose)
man_eyeline
Camera staticMotion smallSize mediumArea mediumCount singleKind person
source 2020×1136 · 24.15 fps · 156 frames · 6.5 s
PRETRAINED_PI3X: inference 52 s (peak 13 GB) · metric scale 0.29 · geometric-mean depth 4.17 · subject depth 1.11 · translation 0.06 / jitter 0.08 % · rotation 0.03 / jitter 0.04° · focal 523 px / jitter 0.06 % · whole-frame depth jitter 1.06 % · region depth jitter static / dyn. 0.04 / 0.10 % · per-pixel depth jitter static / dyn. 0.11 / 0.18 % · per-pixel world jitter static / dyn. 0.14 / 0.32 % (frames with enough tracks: static 100% / dyn. 100%)
V1_PLAIN_FINETUNE: inference 52 s (peak 13 GB) · metric scale 0.27 · geometric-mean depth 5.34 · subject depth 1.54 · translation 0.06 / jitter 0.08 % · rotation 0.03 / jitter 0.03° · focal 516 px / jitter 0.07 % · whole-frame depth jitter 0.96 % · region depth jitter static / dyn. 0.05 / 0.11 % · per-pixel depth jitter static / dyn. 0.14 / 0.19 % · per-pixel world jitter static / dyn. 0.16 / 0.31 % (frames with enough tracks: static 100% / dyn. 100%)
V2_3D_MOTION_LOSS: inference 51 s (peak 13 GB) · metric scale 0.27 · geometric-mean depth 5.33 · subject depth 1.55 · translation 0.05 / jitter 0.07 % · rotation 0.03 / jitter 0.03° · focal 516 px / jitter 0.05 % · whole-frame depth jitter 1.02 % · region depth jitter static / dyn. 0.04 / 0.11 % · per-pixel depth jitter static / dyn. 0.12 / 0.17 % · per-pixel world jitter static / dyn. 0.13 / 0.25 % (frames with enough tracks: static 100% / dyn. 100%)
V3_2D_REPROJ_LOSS: inference 52 s (peak 13 GB) · metric scale 0.28 · geometric-mean depth 5.26 · subject depth 1.50 · translation 0.05 / jitter 0.06 % · rotation 0.03 / jitter 0.03° · focal 518 px / jitter 0.05 % · whole-frame depth jitter 0.36 % · region depth jitter static / dyn. 0.04 / 0.10 % · per-pixel depth jitter static / dyn. 0.12 / 0.16 % · per-pixel world jitter static / dyn. 0.13 / 0.24 % (frames with enough tracks: static 100% / dyn. 100%)
V4_2ND_DEV_CAM_LOSS: inference 51 s (peak 13 GB) · metric scale 0.27 · geometric-mean depth 5.29 · subject depth 1.50 · translation 0.04 / jitter 0.03 % · rotation 0.03 / jitter 0.02° · focal 519 px / jitter 0.03 % · whole-frame depth jitter 0.27 % · region depth jitter static / dyn. 0.02 / 0.07 % · per-pixel depth jitter static / dyn. 0.10 / 0.14 % · per-pixel world jitter static / dyn. 0.13 / 0.20 % (frames with enough tracks: static 100% / dyn. 100%)
(controls all videos of this card)
Depth
Depth only (own camera coordinates, fixed oblique 30° view, no pose)
man_park
Camera mediumMotion mediumSize mediumArea mediumCount crowdKind person + animal
source 2020×1136 · 24.05 fps · 477 frames · 19.8 s
PRETRAINED_PI3X: inference 435 s (peak 16 GB) · metric scale 0.33 · geometric-mean depth 8.69 · subject depth 1.62 · translation 0.11 / jitter 0.12 % · rotation 0.16 / jitter 0.05° · focal 829 px / jitter 0.10 % · whole-frame depth jitter 0.58 % · region depth jitter static / dyn. 0.09 / 0.22 % · per-pixel depth jitter static / dyn. 0.33 / 0.34 % · per-pixel world jitter static / dyn. 0.37 / 0.68 % (frames with enough tracks: static 100% / dyn. 100%)
V1_PLAIN_FINETUNE: inference 427 s (peak 16 GB) · metric scale 0.29 · geometric-mean depth 12.04 · subject depth 2.76 · translation 0.11 / jitter 0.13 % · rotation 0.16 / jitter 0.07° · focal 821 px / jitter 0.10 % · whole-frame depth jitter 0.68 % · region depth jitter static / dyn. 0.11 / 0.22 % · per-pixel depth jitter static / dyn. 0.38 / 0.35 % · per-pixel world jitter static / dyn. 0.41 / 0.60 % (frames with enough tracks: static 100% / dyn. 100%)
V2_3D_MOTION_LOSS: inference 428 s (peak 16 GB) · metric scale 0.29 · geometric-mean depth 11.82 · subject depth 2.63 · translation 0.09 / jitter 0.09 % · rotation 0.16 / jitter 0.05° · focal 815 px / jitter 0.09 % · whole-frame depth jitter 0.62 % · region depth jitter static / dyn. 0.08 / 0.18 % · per-pixel depth jitter static / dyn. 0.31 / 0.28 % · per-pixel world jitter static / dyn. 0.33 / 0.41 % (frames with enough tracks: static 100% / dyn. 100%)
V3_2D_REPROJ_LOSS: inference 427 s (peak 16 GB) · metric scale 0.30 · geometric-mean depth 11.59 · subject depth 2.62 · translation 0.09 / jitter 0.08 % · rotation 0.16 / jitter 0.05° · focal 813 px / jitter 0.09 % · whole-frame depth jitter 0.62 % · region depth jitter static / dyn. 0.08 / 0.16 % · per-pixel depth jitter static / dyn. 0.31 / 0.27 % · per-pixel world jitter static / dyn. 0.33 / 0.40 % (frames with enough tracks: static 100% / dyn. 100%)
V4_2ND_DEV_CAM_LOSS: inference 425 s (peak 16 GB) · metric scale 0.29 · geometric-mean depth 11.82 · subject depth 2.62 · translation 0.08 / jitter 0.06 % · rotation 0.16 / jitter 0.04° · focal 813 px / jitter 0.07 % · whole-frame depth jitter 0.54 % · region depth jitter static / dyn. 0.07 / 0.15 % · per-pixel depth jitter static / dyn. 0.30 / 0.26 % · per-pixel world jitter static / dyn. 0.32 / 0.35 % (frames with enough tracks: static 100% / dyn. 100%)
(controls all videos of this card)
Depth
Depth only (own camera coordinates, fixed oblique 30° view, no pose)
man_winter
Camera mediumMotion largeSize mediumArea mediumCount crowdKind person
source 2020×1136 · 24.07 fps · 339 frames · 14.1 s
PRETRAINED_PI3X: inference 223 s (peak 15 GB) · metric scale 0.31 · geometric-mean depth 4.77 · subject depth 1.61 · translation 0.20 / jitter 0.16 % · rotation 0.09 / jitter 0.04° · focal 1236 px / jitter 0.14 % · whole-frame depth jitter 1.52 % · region depth jitter static / dyn. 0.11 / 0.19 % · per-pixel depth jitter static / dyn. 0.43 / 0.31 % · per-pixel world jitter static / dyn. 0.44 / 0.47 % (frames with enough tracks: static 100% / dyn. 100%)
V1_PLAIN_FINETUNE: inference 219 s (peak 15 GB) · metric scale 0.28 · geometric-mean depth 7.43 · subject depth 3.09 · translation 0.25 / jitter 0.20 % · rotation 0.09 / jitter 0.06° · focal 1212 px / jitter 0.16 % · whole-frame depth jitter 1.37 % · region depth jitter static / dyn. 0.13 / 0.19 % · per-pixel depth jitter static / dyn. 0.48 / 0.31 % · per-pixel world jitter static / dyn. 0.50 / 0.46 % (frames with enough tracks: static 100% / dyn. 100%)
V2_3D_MOTION_LOSS: inference 218 s (peak 15 GB) · metric scale 0.28 · geometric-mean depth 7.42 · subject depth 3.04 · translation 0.21 / jitter 0.14 % · rotation 0.09 / jitter 0.05° · focal 1200 px / jitter 0.13 % · whole-frame depth jitter 1.37 % · region depth jitter static / dyn. 0.10 / 0.17 % · per-pixel depth jitter static / dyn. 0.40 / 0.25 % · per-pixel world jitter static / dyn. 0.41 / 0.31 % (frames with enough tracks: static 100% / dyn. 100%)
V3_2D_REPROJ_LOSS: inference 220 s (peak 15 GB) · metric scale 0.29 · geometric-mean depth 7.21 · subject depth 2.98 · translation 0.21 / jitter 0.13 % · rotation 0.09 / jitter 0.04° · focal 1165 px / jitter 0.11 % · whole-frame depth jitter 1.29 % · region depth jitter static / dyn. 0.11 / 0.17 % · per-pixel depth jitter static / dyn. 0.40 / 0.25 % · per-pixel world jitter static / dyn. 0.41 / 0.30 % (frames with enough tracks: static 100% / dyn. 100%)
V4_2ND_DEV_CAM_LOSS: inference 223 s (peak 15 GB) · metric scale 0.28 · geometric-mean depth 7.23 · subject depth 2.93 · translation 0.19 / jitter 0.08 % · rotation 0.08 / jitter 0.03° · focal 1162 px / jitter 0.09 % · whole-frame depth jitter 1.26 % · region depth jitter static / dyn. 0.10 / 0.14 % · per-pixel depth jitter static / dyn. 0.39 / 0.23 % · per-pixel world jitter static / dyn. 0.40 / 0.25 % (frames with enough tracks: static 100% / dyn. 100%)
(controls all videos of this card)
Depth
Depth only (own camera coordinates, fixed oblique 30° view, no pose)
man_standing_rotate
Camera unclassifiedMotion unclassifiedSize unclassifiedArea unclassifiedCount unclassifiedKind unclassified
source 3840×2160 · 29.97 fps · 330 frames · 11.0 s
PRETRAINED_PI3X: inference 209 s (peak 15 GB) · metric scale 0.35 · geometric-mean depth 7.48 · subject depth 0.64 · translation 0.33 / jitter 0.41 % · rotation 1.09 / jitter 0.21° · focal 542 px / jitter 0.23 % · whole-frame depth jitter 1.61 % · region depth jitter static / dyn. 0.26 / 0.57 % · per-pixel depth jitter static / dyn. 0.95 / 0.68 % · per-pixel world jitter static / dyn. 1.09 / 5.59 % (frames with enough tracks: static 99% / dyn. 100%)
V1_PLAIN_FINETUNE: inference 210 s (peak 15 GB) · metric scale 0.30 · geometric-mean depth 7.76 · subject depth 0.95 · translation 0.65 / jitter 0.90 % · rotation 1.09 / jitter 0.25° · focal 535 px / jitter 0.33 % · whole-frame depth jitter 2.72 % · region depth jitter static / dyn. 0.38 / 0.72 % · per-pixel depth jitter static / dyn. 1.00 / 0.82 % · per-pixel world jitter static / dyn. 1.19 / 8.54 % (frames with enough tracks: static 99% / dyn. 100%)
V2_3D_MOTION_LOSS: inference 212 s (peak 15 GB) · metric scale 0.30 · geometric-mean depth 7.70 · subject depth 0.95 · translation 0.44 / jitter 0.51 % · rotation 1.09 / jitter 0.22° · focal 540 px / jitter 0.27 % · whole-frame depth jitter 2.83 % · region depth jitter static / dyn. 0.27 / 0.55 % · per-pixel depth jitter static / dyn. 0.81 / 0.66 % · per-pixel world jitter static / dyn. 0.97 / 4.85 % (frames with enough tracks: static 99% / dyn. 100%)
V3_2D_REPROJ_LOSS: inference 210 s (peak 15 GB) · metric scale 0.30 · geometric-mean depth 6.85 · subject depth 0.90 · translation 0.42 / jitter 0.45 % · rotation 1.09 / jitter 0.20° · focal 539 px / jitter 0.25 % · whole-frame depth jitter 2.79 % · region depth jitter static / dyn. 0.27 / 0.51 % · per-pixel depth jitter static / dyn. 0.81 / 0.62 % · per-pixel world jitter static / dyn. 0.97 / 4.10 % (frames with enough tracks: static 99% / dyn. 100%)
V4_2ND_DEV_CAM_LOSS: inference 209 s (peak 15 GB) · metric scale 0.30 · geometric-mean depth 7.15 · subject depth 0.92 · translation 0.36 / jitter 0.35 % · rotation 1.09 / jitter 0.17° · focal 540 px / jitter 0.19 % · whole-frame depth jitter 2.62 % · region depth jitter static / dyn. 0.24 / 0.45 % · per-pixel depth jitter static / dyn. 0.81 / 0.57 % · per-pixel world jitter static / dyn. 0.94 / 3.29 % (frames with enough tracks: static 99% / dyn. 100%)
(controls all videos of this card)
Depth
Depth only (own camera coordinates, fixed oblique 30° view, no pose)
breakdance (frames 590-1180)
Camera mediumMotion largeSize mediumArea smallCount singleKind person
source 1920×1080 · 24.00 fps · 590 frames · 24.6 s
PRETRAINED_PI3X: inference 643 s (peak 19 GB) · metric scale 0.30 · geometric-mean depth 1.60 · subject depth 1.12 · translation 0.83 / jitter 0.89 % · rotation 0.32 / jitter 0.16° · focal 410 px / jitter 0.23 % · whole-frame depth jitter 1.26 % · region depth jitter static / dyn. 0.36 / 0.67 % · per-pixel depth jitter static / dyn. 0.45 / 0.97 % · per-pixel world jitter static / dyn. 1.25 / 1.70 % (frames with enough tracks: static 100% / dyn. 99%)
V1_PLAIN_FINETUNE: inference 589 s (peak 19 GB) · metric scale 0.29 · geometric-mean depth 2.43 · subject depth 1.79 · translation 0.85 / jitter 0.96 % · rotation 0.30 / jitter 0.18° · focal 434 px / jitter 0.24 % · whole-frame depth jitter 1.14 % · region depth jitter static / dyn. 0.36 / 0.65 % · per-pixel depth jitter static / dyn. 0.47 / 0.96 % · per-pixel world jitter static / dyn. 1.41 / 1.69 % (frames with enough tracks: static 100% / dyn. 99%)
V2_3D_MOTION_LOSS: inference 657 s (peak 19 GB) · metric scale 0.29 · geometric-mean depth 2.48 · subject depth 1.78 · translation 0.75 / jitter 0.69 % · rotation 0.29 / jitter 0.15° · focal 436 px / jitter 0.22 % · whole-frame depth jitter 1.10 % · region depth jitter static / dyn. 0.33 / 0.61 % · per-pixel depth jitter static / dyn. 0.42 / 0.90 % · per-pixel world jitter static / dyn. 0.93 / 1.34 % (frames with enough tracks: static 100% / dyn. 99%)
V3_2D_REPROJ_LOSS: inference 650 s (peak 19 GB) · metric scale 0.29 · geometric-mean depth 2.38 · subject depth 1.74 · translation 0.77 / jitter 0.68 % · rotation 0.30 / jitter 0.15° · focal 430 px / jitter 0.23 % · whole-frame depth jitter 1.21 % · region depth jitter static / dyn. 0.34 / 0.64 % · per-pixel depth jitter static / dyn. 0.43 / 0.91 % · per-pixel world jitter static / dyn. 0.93 / 1.30 % (frames with enough tracks: static 100% / dyn. 99%)
V4_2ND_DEV_CAM_LOSS: inference 645 s (peak 19 GB) · metric scale 0.29 · geometric-mean depth 2.45 · subject depth 1.77 · translation 0.68 / jitter 0.44 % · rotation 0.29 / jitter 0.11° · focal 427 px / jitter 0.17 % · whole-frame depth jitter 1.04 % · region depth jitter static / dyn. 0.27 / 0.58 % · per-pixel depth jitter static / dyn. 0.37 / 0.86 % · per-pixel world jitter static / dyn. 0.68 / 1.12 % (frames with enough tracks: static 100% / dyn. 99%)
(controls all videos of this card)
Depth
Depth only (own camera coordinates, fixed oblique 30° view, no pose)
| Video | Model | Frames | Length (s) | Translation speed % | Translation jitter % | Rotation speed ° | Rotation jitter ° | Focal jitter % | Whole-frame depth jitter % | Region depth jitter % static / dyn. | Per-pixel depth jitter % static / dyn. | Per-pixel world jitter % static / dyn. | Dynamic-object share | Valid points |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| man_airport | PRETRAINED_PI3X | 636 | 26.5 | 0.32 | 0.47 | 0.08 | 0.08 | 0.25 | 1.59 | 0.19 / 0.25 | 0.33 / 0.47 | 0.41 / 1.36 | 41% | 78% |
| man_airport | V1_PLAIN_FINETUNE | 636 | 26.5 | 0.34 | 0.52 | 0.08 | 0.08 | 0.28 | 1.41 | 0.23 / 0.28 | 0.38 / 0.51 | 0.47 / 1.35 | 41% | 72% |
| man_airport | V2_3D_MOTION_LOSS | 636 | 26.5 | 0.26 | 0.39 | 0.07 | 0.06 | 0.22 | 1.31 | 0.18 / 0.24 | 0.32 / 0.46 | 0.36 / 1.03 | 41% | 73% |
| man_airport | V3_2D_REPROJ_LOSS | 636 | 26.5 | 0.23 | 0.34 | 0.07 | 0.06 | 0.20 | 1.25 | 0.17 / 0.24 | 0.32 / 0.46 | 0.35 / 0.90 | 41% | 73% |
| man_airport | V4_2ND_DEV_CAM_LOSS | 636 | 26.5 | 0.17 | 0.22 | 0.07 | 0.04 | 0.14 | 1.14 | 0.13 / 0.21 | 0.29 / 0.43 | 0.31 / 0.69 | 41% | 74% |
| man_eyeline | PRETRAINED_PI3X | 156 | 6.5 | 0.06 | 0.08 | 0.03 | 0.04 | 0.06 | 1.06 | 0.04 / 0.10 | 0.11 / 0.18 | 0.14 / 0.32 | 19% | 93% |
| man_eyeline | V1_PLAIN_FINETUNE | 156 | 6.5 | 0.06 | 0.08 | 0.03 | 0.03 | 0.07 | 0.96 | 0.05 / 0.11 | 0.14 / 0.19 | 0.16 / 0.31 | 19% | 91% |
| man_eyeline | V2_3D_MOTION_LOSS | 156 | 6.5 | 0.05 | 0.07 | 0.03 | 0.03 | 0.05 | 1.02 | 0.04 / 0.11 | 0.12 / 0.17 | 0.13 / 0.25 | 19% | 92% |
| man_eyeline | V3_2D_REPROJ_LOSS | 156 | 6.5 | 0.05 | 0.06 | 0.03 | 0.03 | 0.05 | 0.36 | 0.04 / 0.10 | 0.12 / 0.16 | 0.13 / 0.24 | 19% | 93% |
| man_eyeline | V4_2ND_DEV_CAM_LOSS | 156 | 6.5 | 0.04 | 0.03 | 0.03 | 0.02 | 0.03 | 0.27 | 0.02 / 0.07 | 0.10 / 0.14 | 0.13 / 0.20 | 19% | 93% |
| man_park | PRETRAINED_PI3X | 477 | 19.8 | 0.11 | 0.12 | 0.16 | 0.05 | 0.10 | 0.58 | 0.09 / 0.22 | 0.33 / 0.34 | 0.37 / 0.68 | 21% | 72% |
| man_park | V1_PLAIN_FINETUNE | 477 | 19.8 | 0.11 | 0.13 | 0.16 | 0.07 | 0.10 | 0.68 | 0.11 / 0.22 | 0.38 / 0.35 | 0.41 / 0.60 | 21% | 70% |
| man_park | V2_3D_MOTION_LOSS | 477 | 19.8 | 0.09 | 0.09 | 0.16 | 0.05 | 0.09 | 0.62 | 0.08 / 0.18 | 0.31 / 0.28 | 0.33 / 0.41 | 21% | 71% |
| man_park | V3_2D_REPROJ_LOSS | 477 | 19.8 | 0.09 | 0.08 | 0.16 | 0.05 | 0.09 | 0.62 | 0.08 / 0.16 | 0.31 / 0.27 | 0.33 / 0.40 | 21% | 71% |
| man_park | V4_2ND_DEV_CAM_LOSS | 477 | 19.8 | 0.08 | 0.06 | 0.16 | 0.04 | 0.07 | 0.54 | 0.07 / 0.15 | 0.30 / 0.26 | 0.32 / 0.35 | 21% | 71% |
| man_winter | PRETRAINED_PI3X | 339 | 14.1 | 0.20 | 0.16 | 0.09 | 0.04 | 0.14 | 1.52 | 0.11 / 0.19 | 0.43 / 0.31 | 0.44 / 0.47 | 38% | 67% |
| man_winter | V1_PLAIN_FINETUNE | 339 | 14.1 | 0.25 | 0.20 | 0.09 | 0.06 | 0.16 | 1.37 | 0.13 / 0.19 | 0.48 / 0.31 | 0.50 / 0.46 | 38% | 60% |
| man_winter | V2_3D_MOTION_LOSS | 339 | 14.1 | 0.21 | 0.14 | 0.09 | 0.05 | 0.13 | 1.37 | 0.10 / 0.17 | 0.40 / 0.25 | 0.41 / 0.31 | 38% | 61% |
| man_winter | V3_2D_REPROJ_LOSS | 339 | 14.1 | 0.21 | 0.13 | 0.09 | 0.04 | 0.11 | 1.29 | 0.11 / 0.17 | 0.40 / 0.25 | 0.41 / 0.30 | 38% | 60% |
| man_winter | V4_2ND_DEV_CAM_LOSS | 339 | 14.1 | 0.19 | 0.08 | 0.08 | 0.03 | 0.09 | 1.26 | 0.10 / 0.14 | 0.39 / 0.23 | 0.40 / 0.25 | 38% | 61% |
| man_standing_rotate | PRETRAINED_PI3X | 330 | 11.0 | 0.33 | 0.41 | 1.09 | 0.21 | 0.23 | 1.61 | 0.26 / 0.57 | 0.95 / 0.68 | 1.09 / 5.59 | 14% | 53% |
| man_standing_rotate | V1_PLAIN_FINETUNE | 330 | 11.0 | 0.65 | 0.90 | 1.09 | 0.25 | 0.33 | 2.72 | 0.38 / 0.72 | 1.00 / 0.82 | 1.19 / 8.54 | 14% | 43% |
| man_standing_rotate | V2_3D_MOTION_LOSS | 330 | 11.0 | 0.44 | 0.51 | 1.09 | 0.22 | 0.27 | 2.83 | 0.27 / 0.55 | 0.81 / 0.66 | 0.97 / 4.85 | 14% | 43% |
| man_standing_rotate | V3_2D_REPROJ_LOSS | 330 | 11.0 | 0.42 | 0.45 | 1.09 | 0.20 | 0.25 | 2.79 | 0.27 / 0.51 | 0.81 / 0.62 | 0.97 / 4.10 | 14% | 42% |
| man_standing_rotate | V4_2ND_DEV_CAM_LOSS | 330 | 11.0 | 0.36 | 0.35 | 1.09 | 0.17 | 0.19 | 2.62 | 0.24 / 0.45 | 0.81 / 0.57 | 0.94 / 3.29 | 14% | 43% |
| breakdance (frames 590-1180) | PRETRAINED_PI3X | 590 | 24.6 | 0.83 | 0.89 | 0.32 | 0.16 | 0.23 | 1.26 | 0.36 / 0.67 | 0.45 / 0.97 | 1.25 / 1.70 | 10% | 48% |
| breakdance (frames 590-1180) | V1_PLAIN_FINETUNE | 590 | 24.6 | 0.85 | 0.96 | 0.30 | 0.18 | 0.24 | 1.14 | 0.36 / 0.65 | 0.47 / 0.96 | 1.41 / 1.69 | 10% | 48% |
| breakdance (frames 590-1180) | V2_3D_MOTION_LOSS | 590 | 24.6 | 0.75 | 0.69 | 0.29 | 0.15 | 0.22 | 1.10 | 0.33 / 0.61 | 0.42 / 0.90 | 0.93 / 1.34 | 10% | 48% |
| breakdance (frames 590-1180) | V3_2D_REPROJ_LOSS | 590 | 24.6 | 0.77 | 0.68 | 0.30 | 0.15 | 0.23 | 1.21 | 0.34 / 0.64 | 0.43 / 0.91 | 0.93 / 1.30 | 10% | 48% |
| breakdance (frames 590-1180) | V4_2ND_DEV_CAM_LOSS | 590 | 24.6 | 0.68 | 0.44 | 0.29 | 0.11 | 0.17 | 1.04 | 0.27 / 0.58 | 0.37 / 0.86 | 0.68 / 1.12 | 10% | 49% |