can it do tracking witout detections on every frame ?

[vitalik]

Can trackers tool do tracking without running model inference on every frame ?

def callback(frame, _):
    result = model.infer(frame)[0] # <---- this takes 99% of time
    detections = sv.Detections.from_inference(result)
    detections = tracker.update(detections)

my goal is to track objects on a video close to real time (20-30 fps) while being able to filter what to track using vision model

[rolson24]

Hi @vitalik!

These object trackers are not really designed to skip frames, however if you have slow moving, or objects that don't change direction very often, you might be able to get away with running the tracker every other frame, but it would probably perform much worse.

[Inferno-P]

Hey @vitalik 👋

I had a similar problem. I tried the following methods to overcome this:

1. Frame Skipping [Very low effort]: Run model inference only every N frames. This will result in choppy trackers with gaps - but it's a trade off question subjective to implementation details.

2. Motion Estimation [Medium effort]: Once you're skipping frames, you can use good old Open CV to fill in the gaps .by running Optical Flow.
- TIP : Run it with sparse method. Not dense. Speed boost is roughly 30%. 🔥

3. Confidence-based Updates [Risky ⚠ ]: Only update when detection confidence drops below a threshold. This worked for me only in 'some' cases.