The 3D Particle Tracker was developed for Tomás Saraceno's installation Particular Matter. A software created with Elliot Woods (3D Tracking Core) and Alex Guevara (Visualisation Basis) to track and visualise dust particles in three dimensions.

A stereoscopic pair of high frequency cameras were capturing dust particles floating in an extremely bright light beam. These stereo images were used in a custom made, multi-threaded OpenCV pipeline, eventually tracking many hundreds of particles in real-time at 120 frames per second. The usage of two cameras enabled a triangulation of the particles position into a 3 dimensional path.

The calculated position of the dust particles were visualised using GLSL shaders. The resulting image was projected as part of the exhibit. Additionally, the software sent the 3D tracking data to a spatializing sound engine developed by Studio Tomás Saraceno, sonifying the movement and turbulences inside the particles stream.

Calibration

Key to a successful tracking setup was a precise calibration system. Each camera was calibrated individually, and later on calibrated to each other, calculating the exact positional relation between the cameras' lenses.

Contours → Triangulate → Tracking

The multi-threaded OpenCV pipeline integrated into TouchDesigner consisted of many small dependable steps. One of which was finding the particles' contours and their mass' center on a sub-pixel level. These values were used to calculate epipolar lines, creating a vast amount of triangulated, possible candidates. The Tracking process was used to filter the candidates and finally resulting in real particles being detected and tracked.

Optical Flow

Next to being used in the Tracking process of the OpenCV pipeline, Optical Flow was further used to determine a more macro level stream of motion. These movements were mainly observed to detect turbulences or calculate generalized direction patterns.

The most advanced intergation of C++ into TouchDesigner yet. An overview of the technical backbone.