By Francesco Dorigo, Bao Nguyen & Daniel Howell, Comcast
The IP video analytics platform design described herein streamlines the conversion of event-based analytics telemetry into time series visualization. With this objective in mind, Comcast developed a platform with the flexibility to support a wide variety of data producers and data consumers, and with the scalability to provide an enterprise solution for real-time analytics.
Parametrized and configurable execution libraries automate repetitive data engineering tasks. In addition, our analytics system provides abstraction layers both for data ingress and data egress, which enables a seamless evolution of the ETL (Extract, Transform, Load) pipeline. This has two main advantages: First to simplify the efforts related to upgrading the pipeline by letting producers and consumers evolve at their own pace, and second, the underlying technologies can seamlessly evolve with zero impact for ingestion and aggregation layers (producers and consumers).
This document describes the major components of our IP video analytics data pipeline, with a specific focus on custom components. This design reduces the time between data ingress and insight.