I Asked Optical Flow for a Hand's Speed. Past a Knee It Reports the Wall.

I built a synthetic test to size a small correction, and found the correction was the whole signal.

The project is cyclegraph. It reads Build AI's per-frame "active manipulation" flag over builddotai/Egocentric-10K as duty cycle, one of the two inputs the ACGIH TLV needs for Hand Activity Level. The other input is hand speed. Ten thousand hours of head-mounted factory video, 2,144 workers, and no way to ask any of them how fast their hands were moving. So you estimate it: put a detector on the hand, run dense optical flow, take the median magnitude inside the box, subtract the camera's own motion, convert pixels to millimetres.

That pipeline is ordinary. I wanted its error bar before spending GPU hours on it. What came back was a number that stops describing the hand at all, past a threshold that normal work crosses.

The budget

The HAL mapping I use is Akkas 2015, fitted over hand speeds from 255.3 to 1288.0 mm/s. It is logistic in log speed, so the error you can afford is a percentage, not a quantity. I pre-registered the tolerable systematic error at 0.25 HAL. At an assumed corpus median of 612.4 mm/s that is 26.7 mm/s, or 4.4% of the speed. At 400 mm/s it is 5.7%, at 800 mm/s 4.6%. About five percent, then. Hold that number.

Rotation on a fisheye

A sanity test comes before any real frame. Point the camera at a hand that is not moving, rotate the camera, ask the estimator for the hand's speed, and expect zero. Rotational flow is depth-independent, so a correct ego-motion model cancels it exactly.

On a narrow lens it does. The corpus has no narrow lens. Its cameras are fisheye, Kannala-Brandt with 67.892 degrees of half-angle, published per worker and identical for all 2,144 of them. (I checked sixteen, drawn at an even stride, and their intrinsics.json files were byte-identical; the other workers' lenses are assumed. There is no real per-camera calibration either, so what follows is a floor for the nominal lens, not for the fleet.) Rotational flow on a fisheye varies strongly with radius, and the rule I am implementing subtracts a scalar, the median flow over everything outside the hand box. A scalar cancels a uniform field and leaves a radial one mostly intact.

The residual comes in closed form, no estimator involved. At 10 degrees per second of head rotation the corpus fisheye leaves 10.0916 mm/s of apparent hand speed, and a narrow lens leaves 0.1328 mm/s. That is seventy-six times more from the same subtraction. Against a 26.7 mm/s budget 10 mm/s is uncomfortable but survivable, and head translation is worse, since 3 cm/s exhausts the whole allowance. It still looked like the problem I expected, a geometric error that is bounded and could be subtracted.

Then I ran the estimators.

The sweep

I rendered frame pairs under the same fisheye, hand plane at 0.45 m and background plane at 2.5 m, and swept how far the hand box moves between the two frames. The scene is synthetic, so the true flow is known exactly, which is the only reason any of this is measurable. Both planes carry one texture, the box does not move between frames, and the warp is backward and first-order, accurate to a fraction of a pixel where flow is smooth. At the hand boundary it is not smooth. There is no occlusion, disocclusion, motion blur, rolling shutter, sensor noise or illumination change. The last four are what the project's own red-team document names as the causes of real flow failure. The estimator got an easier problem than a factory would give it.

The quantity is gain: median recovered flow magnitude over median true flow magnitude, inside the box. Gain 1.0 recovers the motion, and 0.2 reports a fifth of it. Farneback at 960x540, seed 11:

I expected degradation, which you can model and correct. Instead gain holds at essentially one up to 22.8 px, falls off a cliff somewhere between 22.8 and 34.2, and then sits on a shelf, and the shelf turned out to be the part worth explaining.

0.18 is 0.45 divided by 2.5, hand plane over background plane. Past the knee the estimator is measuring the background correctly, and the background is 5.6 times further away, so it moves 5.6 times slower in pixels. The window smoothed across the depth discontinuity at the hand's edge, lost the hand and locked onto the wall behind it, which makes the number a real measurement of the wall. The budget was five percent and the shelf under-reports by eighty-two.

One consequence of the setup matters here. The sweep translates the camera over a static scene, which is how a static hand gets relative motion in a render, so the background moves at the depth ratio and a lost hand lands on 0.18. With a static camera and a hand moving on its own, the background would not move and the same fallback would land near zero. The knee is unchanged either way. That the shelf's value holds for head-mounted factory video, which is ego-motion-dominated, is an argument and not a measurement. The two depth planes are assumptions about workstation geometry too. 0.45 m and 2.5 m are reasonable for a bench, the shelf sits at their ratio. A different workstation should give a different shelf. I have not tested that.

A timing detail cuts against me. Both sweeps use 0.25 s between frames, which is what reading the rubric's pair at the 4 Hz analysis rate gives. The project later re-specified the pair to the clip's own frame rate, about 0.033 s. Gain against pixel displacement is unaffected, because the interval only rescales speeds. But at the native rate the assumed median of 612.4 mm/s is roughly 23 px at 960x540, which sits at the Farneback knee and not comfortably past it.

RAFT-small

Farneback is from 2003, so the obvious move is something better. Same sweep, same seed, same scenes:

RAFT holds full gain at 34.2 px, where Farneback has already fallen. On a nine-point grid that buys somewhere between one and two times the usable displacement, and the grid cannot say where in that range. Then it falls to the same shelf: 0.172, 0.177, 0.180, 0.184 against Farneback's 0.177, 0.187, 0.196. The estimator moved the knee, and the shelf stayed where the geometry put it.

The far tail has a second regime I make no claims about. Gain falls below the depth ratio toward zero once the estimator exceeds its search range and tracks nothing, and that arrives earlier at higher resolution, unambiguously by 0.93 of a hand-box width at 1920x1080 against 1.69 at 960x540. One point sits between the regimes and I will not assign it — 0.1056 at 0.47 box widths, too low for the shelf and too high for nothing. Regime-two points are not on the shelf and should not be averaged with the ones that are.

The ego-motion column has a tell. On the rotation-only test the exact geometry leaves 1.1461 px of residual, what a perfect estimator should report, since the scalar median cannot cancel the radial field. Farneback reports 1.2171. RAFT reports 1.014, below the floor an ideal estimator would leave. That is under-recovery, and it happens to point the direction that looks like a better score. Two estimators is two, and their agreement says little about dense optical flow in general and nothing about a method that models depth. The metric is also blind to direction: an estimator returning the exact field negated scores the same as a perfect one. A median hides a field that is right in half the box and wrong in the other half.

Decode resolution

I swept four decode sizes expecting a monotone trade of cost against fidelity. At the same box-relative displacement, 0.236 of the hand box width:

960x540 is a measured optimum. 1920x1080 does worse than 480x270, with sixteen times the pixels, and it has already fallen off the cliff at a displacement where 960x540 is still recovering 99% of the motion.

I do not have a clean account of why. The knee is not fixed in absolute pixels and not fixed as a fraction of the hand box either, since each reading is contradicted by a row of that table. Farneback's window is 15 px and its pyramid is three levels, which sets a search range, but the crossings do not line up with that arithmetic cleanly enough for me to claim it. Anyone choosing a decode resolution on the theory that more pixels means more signal should know the measurement disagrees by a factor of five.

Why nothing noticed

The pipeline subtracts ego-motion as the median flow outside the hand box. Past the knee the box already contains the background's flow, so the correction removes the signal it was meant to clean. The two failures point the same way and cancel, and what comes out is small and plausible, since a hand moving slowly at a factory workstation is not a suspicious reading.

Every pair in the sweep returned a finite, well-formed flow field, and flow_returned_none is false on all 54 distinct estimator, decode and displacement measurements published here. No contract check and no null-rate ceiling downstream has anything to fire on. That is why I wrote it up instead of fixing it and moving on.

None of this says the instrument works. No ergonomist has scored this corpus, force is unobservable from the video, and every record the project has written carries tlv_evaluable false. Any summary claiming cyclegraph applies the TLV is wrong. This is about one input to it.

Running it yourself

My displacement numbers will not transfer to your setup, since the knee moves with the estimator, the decode resolution, the frame interval and the geometry, and I have shown all four moving it. The generator and the sweep are published, and the Farneback arm needs no corpus, no token and no GPU. The harness takes your estimator as an argument and reports the displacement where your gain leaves 1.0, the shelf it lands on, and the depth ratio that shelf should equal if it has switched to the background. Gain will not stay near 1.0 everywhere; what you can check is whether your knee sits outside the displacements your work produces.

• Benchmark and harness: huggingface.co/datasets/caiotheodoro/cyclegraph-flow-gain
• The data behind every table above: huggingface.co/spaces/caiotheodoro/cyclegraph
• Repository, including what is blocked and why: github.com/caiotheodoro/cyclegraph