Why This Matters

Video workloads are naturally parallel.

A deployment may contain several cameras. A single image may contain several people. One event may require several models or reasoning stages.

Processing all of these sequentially can create latency that grows with the complexity of the scene.

Our research explores ways to execute visual workloads concurrently.

Parallelism at Several Levels

Multiple camera streams

Independent video streams can often be decoded, sampled and analysed simultaneously.

Multiple regions or people

A frame can be divided into relevant regions that are processed in parallel.

Multiple models

Different capabilities can analyse the same frame concurrently when hardware resources allow it.

Pipeline stages

Video capture, preprocessing, inference, reasoning and event delivery do not always need to wait for one another.

A Conceptual Pipeline

Camera streams
   ↓
Concurrent decoding
   ↓
Parallel preprocessing
   ↓
Model workers
   ↓
Reasoning / event aggregation
   ↓
Operational output

What We Are Exploring

• GPU workload scheduling.
• Asynchronous processing.
• Dynamic batching.
• Worker pools.
• Multi-process and multi-threaded inference.
• Separate pipelines for detection and reasoning.
• Region-of-interest processing.
• Load distribution across several accelerators.

Multi-Person Scenes

Scenes with several people are particularly important for applications such as PPE.

Instead of sending every person through a heavy processing path sequentially, alternative strategies include:

• Detect people first.
• Create person-specific regions.
• Process several regions concurrently.
• Aggregate the resulting structured outputs.

This can significantly change the latency profile of a system.

Research Challenges

Parallel execution is not automatically faster.

Memory pressure, model loading, GPU contention and synchronisation overhead can remove the expected benefit.

The research challenge is therefore to find the right level of concurrency for each model, hardware profile and operational workload.