By Santhana Chari, Ph.D., OpenVault; Mahesh Kanase, Ksolves
The Cable and Telecommunication industry has been at the forefront of collecting staggering amounts of data given their end-user subscriber base runs into hundreds of millions of users. The data is collected from devices that are deployed in both core and edge of the network, and at consumer residences that are geographically distributed. Large volumes of data thus collected spans various categories ranging from consumer specific data, aggregated network utilization and usage data, and operational data from hardware devices and software micro-services. This data collection has historically used legacy protocols such as simple network management protocol (SNMP) and Internet Protocol Detail Record (IPDR). Most legacy data collection frameworks use pull-models where the collectors periodically poll the devices to collect and aggregate the data. But with increased emphasis on network automation and orchestration driven by distributed access architectures, there is a growing impetus to migrate to more modern modeldriven telemetry approaches where the endpoints are configured to stream the data using push-models that are based on standard specifications in a vendor agnostic fashion.
With the availability of copious amounts of data comes the natural question of effective approaches to leverage the data to optimize network planning and operations. In the past statistical methods and models that were originally invented several decades ago were used to analyze the data to perform diagnostic and predictive analysis. Diagnostic analysis was mainly used to identify root causes of issues in the network based on historical or real-time data. Predictive analysis, on the other hand, used historical data to estimate future load on the network and prepare the network to meet the quality of experience requirements [ulm-2019]. More recently these statistical approaches were replaced by classical Machine Learning (ML) approaches that use classification and regression techniques using supervised and unsupervised learning techniques [volpe-2021], [righetti-2023].
In this paper, we primarily focus on the use of artificial intelligence (AI) tools, and more specifically Generative AI tools to leverage the vast repository of data and to address proactive network management (PNM). Rapid and recent advances in the transformer models [vaswani-2017] have completely changed the paradigm on how a non-technical user can interact with complex software systems employing large language models (LLM) using only simple natural language queries. We have addressed the problem of how LLMs that have been pre-trained for very general tasks can be customized to analyze and leverage data specific to certain domains, in this case the knowledge base and data specific to the cable industry. The rest of the paper is organized as follows. Section 3 provides details on techniques to augment LLMs with private, application-specific data; we discuss and compare two different techniques, namely finetuning and retrieval augmented generation (RAG). We also present details on how to evaluate the efficacy of RAG applications. We address both end-to-end evaluation of RAG applications using a set of pre-defined prompts and expected responses, as well as present techniques to evaluate the individual building blocks of a RAG system, namely, the embedding, chunk size, number of chunks, etc. Finally, we present details on generalized retrievers using the LangChain framework that can leverage local or third-party data to build advanced retrieval systems.