By Bruce E. Barker Jr., Claude Bou Abboud & Erik Neeld, Comcast Cable
As means to control the spread of the COVID-19 virus, many countries implemented measures to quarantine large portions of their populations in early 2020. Remote working and schooling policies were applied to a fairly large extent. Large portions of the workforce, especially in the hospitality industry, were laid off. Essentially, many people found themselves stuck at home for one reason or another.
This resulted in a significant increase in network traffic in many areas for many service providers. This was especially true for upstream traffic generated by various teleconferencing and VoIP-type protocols that were heavily leveraged to support remote working and schooling. Additionally, there was a noted shift in the daily peak time - from evenings to daytime in many locations.
Traffic increases caused an increase in the utilization of all network elements including cable modem termination system (CMTS) service groups (SGs) which provide connectivity between the core network the access network where customer homes and businesses reside. High utilization can result in potential impact to customer experience such as slow speeds, poor video quality, and intermittent service if not resolved. Speed test data and various industry models have helped us to define highly utilized CMTS SGs as those with utilization exceeding 85% consistently.
Sharp increases in traffic and CMTS SG utilization like what was observed during the COVID-19 event can be well-managed through a combination of proactive and reactive capacity management activities.
Proactive measures are mainly focused on predictive traffic forecasting and capacity deployment planning. Reactive measures include a plethora of specialized reporting mechanisms, to help prioritize near-term capacity work, and adaptations to existing long-term plans where applicable.
Accelerated development of new solutions, techniques, and technologies is also a key aspect of any strategy during a crisis. To expand upstream capacity, development and testing for additional upstream channels was prioritized and successfully deployed. The profile management application (PMA) platform, known by the name “Octave” at Comcast and initially developed for DOCSIS 3.1 downstream capacity, was rapidly modified to enable more DOCSIS 3.0 upstream capacity. To help operationalize and deploy new solutions, other tools and automation were also quickly developed and modified.
Furthermore, the partnership between capacity management, engineering, finance, and technical operations teams on the planning and execution of capacity solutions was critical to tackling the challenge imposed by the sudden network growth. It was only through this partnership and alignment that extraordinary innovations and accomplishments were made. Node split work to mitigate highly utilized CMTS SGs, for example, increased 100% from normal levels as all teams rallied around a common goal.
This would not have been possible otherwise. All efforts to manage capacity in extreme circumstances like during COVID-19 are only effective when there is good partnership and solid alignment across the organization.