Why OnsetBench

Measuring whether AI can recover physical events

Haptera
October 2026

Watch a robot place a part in a fixture. You may be able to describe the task without being able to tell exactly when the part starts pressing against the fixture. A model describing the same video faces that question too.

The distinction matters when someone uses the model’s output. A researcher might use a contact timestamp to train a robot. An engineer might use it to investigate a failed placement. Getting the event or its timing wrong can lead either of them to associate contact with the wrong action.

We built OnsetBench to test how well systems recover physical events from video. We’re starting with contact and its timing, comparing predictions with synchronized measurements from instruments that recorded the interaction.[1]

What counts as contact?

Before we can score an answer, we have to agree on the question.

Was contact present? When did it begin? A model can recognize that a robot is pressing against something and still get the start time wrong. We test those separately.

We also have to be clear about what the instrument measures. A force sensor might register loading only after it crosses a threshold. A light touch may never cross that threshold, and contact elsewhere may not register at all. Readers need to know which of those events the test covers.

Then there are the clocks. Video, robot telemetry, and force measurements may have different timestamps. We need to know how they line up before judging whether a prediction was early or late. Our timing conclusions can only be as precise as that alignment allows.

What does the score tell us?

A model can be right almost every time it reports contact and still miss most of the contact in a recording. Another model might find more events but make too many false calls.

We need to count what each model gets right, what it gets wrong, what it misses, and how often it declines to answer. The settings used to produce those results matter too: a model that answers only when it is highly confident is making a different tradeoff from one that answers every time.

“No contact” and “we can’t tell” must stay separate. Counting an uncertain answer as “no contact” puts a claim into the record that the model did not establish. We also need to check whether the model’s confidence matches how often it is actually right.

Timing results can hide a similar problem. Reporting the timing error only for events a model successfully found leaves out everything it missed. We need to see how many events it found within the allowed timing error and how many predictions had no matching reference event per hour of recording.

What the model gets to see changes the test as well. Giving it a short clip of one task provides context it would have to find for itself in a full recording. Letting it look at frames from after an event may help it identify when that event began. That result does not tell us how quickly it could report the event as it happens.[1]

Will it work somewhere else?

A result from one setup leaves questions about the next one. What happens with a different object, task, camera angle, or facility?

We need recordings that were kept out of training and model selection. Closely related examples should stay together so information does not leak between development and evaluation. When recordings have been used before, that history needs to be disclosed.

Even then, testing a new recording from a familiar setup is different from testing a new environment. Adding more training data does not, by itself, answer whether the model will transfer.

When a model fails under a new condition, that helps us understand what it was relying on and where more work is needed.

Does it help?

Finding an event correctly is a useful result. Whether that result helps someone do a job is a separate question.

For robot learning, we need a controlled comparison showing whether the labels improve learned behavior. For failure investigation, we need to see whether the record helps a team understand the problem or check whether a change worked. A benchmark score cannot answer those questions on its own.

Haptera develops OnsetBench and contributes a system to the evaluation. Readers should be able to inspect how we defined the events, what each system received, and how we scored its answers. The benchmark should help them choose an approach even when another system performs better, a simple rule using the allowed inputs is enough, or none of the tested methods meets their needs.

That is what we want OnsetBench to provide: a way to check what a system can tell us about a physical interaction before we rely on its answer.

Notes and references

[1] Haptera, Onset Benchmark Review v4, 12 September 2026, sections 1–4. See the benchmark results for the tested systems, input settings, scores, and limitations.