Optimizing radio cell quality for capacity and quality of service using machine learning techniques

This application is a reissue of U.S. Pat. No. 10,966,108, issued on Mar. 30, 2021, which claims priority to U.S. patent application Ser. No. 16/033,017, filed on Jul. 11, 2018, which patent and application are incorporated herein by reference in their entirety.

Embodiments of the present invention relate generally to telecommunications, and specifically to optimizing radio cell quality for capacity and quality of service using machine learning techniques.

Typically network operations have a variety of hardware and software systems for managing their networks. For example, network operator's Radio Access Network (RAN) operations and performance departments monitor and optimize their networks using known in the art systems for performance management which may utilize Network Equipment Manufacturer (NEM) cell based counters. The use of the geo-locating subscriber Radio Resource Control Signaling (RRC), Media Access Control (MAC) and radio frequency (RF) conditions found in measurement reports and MAC channel feedback typically allow improvement in prior knowledge down to sub-cell granularity levels. Such geo-located information is typically rendered in a geo-graphical user interface. However, many troubleshooting, parameter tuning and small cell planning approaches require manual so called ‘best guess’ methods. In many cases carrier revenue is deteriorating due to revenue loss to ‘over the top services’, such as, but not limited to, Facebook, Skype, Twitter and the like, increasingly making such carriers last mile telecommunication providers. Consequently, there is a need for carriers to dramatically reduce their operating expenses in order to remain viable and profitable.

One representative solution for reduction of carriers' operating expenses is employment of Self-Organizing Network (SON) functions. SON functions are functions that automatically monitor network operations and parameters, and user actions, and automatically take actions to optimize network operations. SON functions are typically structured hierarchically—with a high-level SON function (for example, a Network Manager (NM)) operating to optimize the overall network, and a plurality of lower-level SON functions (for example, at each Network Element (NE)) locally optimizing the NE, under the control and direction of the high-level SON function.

For many telecommunication carriers there is a need to advance implementation of SON functions of automatic network monitoring operations beyond neighbor cell management.

The purpose and advantages of the illustrated embodiments will be set forth in and apparent from the description that follows. Additional advantages of the illustrated embodiments will be realized and attained by the devices, systems and methods particularly pointed out in the written description and claims hereof, as well as from the appended drawings.

In accordance with a purpose of the illustrated embodiments, in one aspect, a method for optimizing a radio access network includes receiving at least one area of the radio access network to be analyzed from a user and receiving a desired outcome from a user. A plurality of network monitoring parameters related to a user requested analysis is identified. The identified plurality of network monitoring parameters is correlated. A root cause analysis is performed using an automated classification model based on the correlated plurality of network monitoring parameters. A recommendation related to the desired outcome is generated based on the performed root cause analysis.

In another aspect, a system for optimizing a radio access network includes a processor and a memory device coupled to the processor. The memory device contains a set of instructions that, when executed by the processor, cause the processor to receive at least one area of the radio access network to be analyzed from a user and receive a desired outcome from a user and to identify a plurality of network monitoring parameters related to a user requested analysis. The set of instructions, when executed by the processor further cause the processor to correlate the identified plurality of network monitoring parameters and to perform a root cause analysis using an automated classification model based on the correlated plurality of network monitoring parameters. The set of instructions, when executed by the processor also cause the processor to generate a recommendation related to the desired outcome based on the performed root cause analysis.

The present invention is now described more fully with reference to the accompanying drawings, in which illustrated embodiments of the present invention are shown wherein like reference numerals identify like elements. The present invention is not limited in any way to the illustrated embodiments as the illustrated embodiments described below are merely exemplary of the invention, which can be embodied in various forms, as appreciated by one skilled in the art. Therefore, it is to be understood that any structural and functional details disclosed herein are not to be interpreted as limiting, but merely as a basis for the claims and as a representative for teaching one skilled in the art to variously employ the present invention. Furthermore, the terms and phrases used herein are not intended to be limiting but rather to provide an understandable description of the invention.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Although any methods and materials similar or equivalent to those described herein can also be used in the practice or testing of the present invention, exemplary methods and materials are now described. It must be noted that as used herein and in the appended claims, the singular forms “a”, “an,” and “the” include plural referents unless the context clearly dictates otherwise. Thus, for example, reference to “a stimulus” includes a plurality of such stimuli and reference to “the signal” includes reference to one or more signals and equivalents thereof known to those skilled in the art, and so forth.

Several aspects of telecommunication systems will now be presented with reference to various apparatus and methods. These apparatus and methods will be described in the following detailed description and illustrated in the accompanying drawings by various blocks, components, circuits, processes, algorithms, etc. (collectively referred to as “elements”). These elements may be implemented using electronic hardware, computer software, or any combination thereof. Whether such elements are implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system.

By way of example, an element, or any portion of an element, or any combination of elements may be implemented as a “processing system” that includes one or more processors. Examples of processors include microprocessors, microcontrollers, graphics processing units (GPUs), central processing units (CPUs), application processors, digital signal processors (DSPs), reduced instruction set computing (RISC) processors, systems on a chip (SoC), baseband processors, field programmable gate arrays (FPGAs), programmable logic devices (PLDs), state machines, gated logic, discrete hardware circuits, and other suitable hardware configured to perform the various functionality described throughout this disclosure. One or more processors in the processing system may execute software. Software shall be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software components, applications, software applications, software packages, routines, subroutines, objects, executables, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise.

Accordingly, in one or more example embodiments, the functions described may be implemented in hardware, software, or any combination thereof. If implemented in software, the functions may be stored on or encoded as one or more instructions or code on a computer-readable medium. Computer-readable media includes computer storage media. Storage media may be any available media that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can comprise a random-access memory (RAM), a read-only memory (ROM), an electrically erasable programmable ROM (EEPROM), optical disk storage, magnetic disk storage, other magnetic storage devices, combinations of the aforementioned types of computer-readable media, or any other medium that can be used to store computer executable code in the form of instructions or data structures that can be accessed by a computer.

1 FIG. 100 102 104 130 102 is a diagram illustrating an example of a wireless communications system and an access network. The wireless communications system (also referred to as a wireless wide area network (WWAN)) includes base stations, UEs, and an Evolved Packet Core (EPC). The base stationsmay include macro cells (high power cellular base station) and/or small cells (low power cellular base station). The macro cells include base stations. The small cells include femtocells, picocells, and microcells.

102 130 110 102 102 130 112 112 The base stations(collectively referred to as Evolved Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access Network (E-UTRAN)) interface with the EPCthrough backhaul links(e.g., S1 interface). In addition to other functions, the base stationsmay perform one or more of the following functions: transfer of user data, radio channel ciphering and deciphering, integrity protection, header compression, mobility control functions (e.g., handover, dual connectivity), inter-cell interference coordination, connection setup and release, load balancing, distribution for non-access stratum (NAS) messages, NAS node selection, synchronization, radio access network (RAN) sharing, multimedia broadcast multicast service (MBMS), subscriber and equipment trace, RAN information management (RIM), paging, positioning, and delivery of warning messages. The base stationsmay communicate directly or indirectly (e.g., through the EPC) with each other over backhaul links(e.g., X2 interface). The backhaul linksmay be wired or wireless.

102 104 102 106 106 102 106 106 102 108 102 104 104 102 102 104 108 102 104 The base stationsmay wirelessly communicate with the UEs. Each of the base stationsmay provide communication coverage for a respective geographic coverage area. There may be overlapping geographic coverage areas. For example, the small cell′ may have a coverage area′ that overlaps the coverage areaof one or more macro base stations. A network that includes both small cell and macro cells may be known as a heterogeneous network. A heterogeneous network may also include Home Evolved Node Bs (eNBs) (HeNBs), which may provide service to a restricted group known as a closed subscriber group (CSG). The communication linksbetween the base stationsand the UEsmay include uplink (UL) (also referred to as reverse link) transmissions from a UEto a base stationand/or downlink (DL) (also referred to as forward link) transmissions from a base stationto a UE. The communication linksmay use multiple-input and multiple-output (MIMO) antenna technology, including spatial multiplexing, beamforming, and/or transmit diversity. The communication links may be through one or more carriers. The base stations/UEsmay use spectrum up to Y MHz (e.g., 5, 10, 15, 20, 100 MHz) bandwidth per carrier allocated in a carrier aggregation of up to a total of Yx MHz (x component carriers) used for transmission in each direction. The carriers may or may not be adjacent to each other. Allocation of carriers may be asymmetric with respect to DL and UL (e.g., more or less carriers may be allocated for DL than for UL). The component carriers may include a primary component carrier and one or more secondary component carriers. A primary component carrier may be referred to as a primary cell (PCell) and a secondary component carrier may be referred to as a secondary cell (SCell).

114 116 118 116 114 The wireless communications system may further include a Wi-Fi access point (AP)in communication with Wi-Fi stations (STAs)via communication linksin a 5 GHz unlicensed frequency spectrum. When communicating in an unlicensed frequency spectrum, the STAs/APmay perform a clear channel assessment (CCA) prior to communicating in order to determine whether the channel is available.

102 102 114 102 The small cell′ may operate in a licensed and/or an unlicensed frequency spectrum. When operating in an unlicensed frequency spectrum, the small cell′ may employ NR and use the same 5 GHz unlicensed frequency spectrum as used by the Wi-Fi AP. The small cell′, employing NR in an unlicensed frequency spectrum, may boost coverage to and/or increase capacity of the access network.

120 104 120 120 120 124 104 The gNodeB (gNB)may operate in millimeter wave (mmW) frequencies and/or near mmW frequencies in communication with the UE. When the gNBoperates in mmW or near mmW frequencies, the gNBmay be referred to as an mmW base station. Extremely high frequency (EHF) is part of the RF in the electromagnetic spectrum. EHF has a range of 30 GHz to 300 GHz and a wavelength between 1 millimeter and 10 millimeters. Radio waves in the band may be referred to as a millimeter wave. Near mmW may extend down to a frequency of 3 GHz with a wavelength of 100 millimeters. The super high frequency (SHF) band extends between 3 GHz and 30 GHz, also referred to as centimeter wave. Communications using the mmW/near mmW radio frequency band has extremely high path loss and a short range. The mmW base stationmay utilize beamformingwith the UEto compensate for the extremely high path loss and short range.

130 132 134 136 138 140 142 132 144 132 104 130 132 136 142 142 142 140 146 146 140 140 138 102 The EPCmay include a Mobility Management Entity (MME), other MMEs, a Serving Gateway, a Multimedia Broadcast Multicast Service (MBMS) Gateway, a Broadcast Multicast Service Center (BM-SC), and a Packet Data Network (PDN) Gateway. The MMEmay be in communication with a Home Subscriber Server (HSS). The MMEis the control node that processes the signaling between the UEsand the EPC. Generally, the MMEprovides bearer and connection management. All user Internet protocol (IP) packets are transferred through the Serving Gateway, which itself is connected to the PDN Gateway. The PDN Gatewayprovides UE IP address allocation as well as other functions. The PDN Gatewayand the BM-SCare connected to the IP Services. The IP Servicesmay include the Internet, an intranet, an IP Multimedia Subsystem (IMS), a PS Streaming Service (PSS), and/or other IP services. The BM-SCmay provide functions for MBMS user service provisioning and delivery. The BM-SCmay serve as an entry point for content provider MBMS transmission, may be used to authorize and initiate MBMS Bearer Services within a public land mobile network (PLMN), and may be used to schedule MBMS transmissions. The MBMS Gatewaymay be used to distribute MBMS traffic to the base stationsbelonging to a Multicast Broadcast Single Frequency Network (MBSFN) area broadcasting a particular service, and may be responsible for session management (start/stop) and for collecting eMBMS related charging information.

102 130 104 104 104 104 The base station may also be referred to as a gNB, Node B, evolved Node B (eNB), an access point, a base transceiver station, a radio base station, a radio transceiver, a transceiver function, a basic service set (BSS), an extended service set (ESS), or some other suitable terminology. The base stationprovides an access point to the EPCfor a UE. Examples of UEsinclude a cellular phone, a smart phone, a session initiation protocol (SIP) phone, a laptop, a personal digital assistant (PDA), a satellite radio, a global positioning system, a multimedia device, a video device, a digital audio player (e.g., MP3 player), a camera, a game console, a tablet, a smart device, a wearable device, a vehicle, an electric meter, a gas pump, a toaster, or any other similar functioning device. Some of the UEsmay be referred to as IoT devices (e.g., parking meter, gas pump, toaster, vehicles, etc.). The UEmay also be referred to as a station, a mobile station, a subscriber station, a mobile unit, a subscriber unit, a wireless unit, a remote unit, a mobile device, a wireless device, a wireless communications device, a remote device, a mobile subscriber station, an access terminal, a mobile terminal, a wireless terminal, a remote terminal, a handset, a user agent, a mobile client, a client, or some other suitable terminology.

130 150 100 150 152 154 156 158 159 1 FIG. Advantageously, various embodiments of the present invention contemplate a monitoring and troubleshooting system that efficiently and accurately evaluates signaling, performance and configuration data of both EPCand RAN to identify at least one cause of reported and/or anticipated network related failures and applies procedural and iterative root cause analysis to provide automatic recommendations for resolving the reported network failures. According to an embodiment of the present invention, a troubleshooting systemmay include, but not limited to, a computer server operatively coupled to each segment of access networkand to an operations support systems (OSS) platform (interchangeably referred to herein as the “network management platform”). At the highest level, the network management platform provides the computer resources required to perform various network management functions, such as billing, customer care, network management, inventory control, maintenance, trouble ticket reporting, surveillance and service provisioning. In some embodiments, the network management platform comprises a plurality of applications, such as performance, fault, configuration and security management applications. As shown in, troubleshooting systemmay communicate with various data repositories maintained by the network management platform, such as, but not limited to network inventory database, network equipment performance database, network equipment configuration database, network signaling database, fault management database, and the like.

152 100 152 152 152 100 100 100 100 100 152 100 100 Network inventorymay be any database capable of storing data relating to access network. Network inventoryincludes network discovered physical inventory, network discovered logical inventory, and planned network inventory. In one embodiment, network inventory databasestores information about network equipment, cell sites, IP transport, or the like. In some embodiments, network inventorymay store physical network information and logical network information. The physical network information specifies the physical components in the access networkand how the physical components are physically connected to one another in the access network. For example, the physical network information may include the number of machines, managed switching elements (e.g. message routers and broadband network gateway), and how the components are physically connected to one another in the monitored access network. The logical network information may specify the logical connections between a set of physical components in the monitored network(e.g., machines) and a mapping of the logical connections across the physical components of the monitored access network. Additionally, network inventorymay store Network Function Virtualization (NFV) information. NFV, as described in the European Telecommunications Standards Institute (ETSI) group specification (GS) NFV 002 v1.1.1, entitled “Network Functions Virtualisation (NFV); Architectural Framework,” published October 2013, which is incorporated herein as if reproduced in its entirety, consolidates many types of physical network devices onto one or more general purpose servers, switches, storage, and/or other general purpose network nodes. For example, NFV may implement network functions performed by a variety of physical network devices that include, but are not limited to switching elements, mobile network nodes, traffic analysis (e.g. deep packet inspection (DPI) and quality of service (QoS) measurement), application level optimization (e.g. application accelerators and content distribution networks (CDNs)), and security functions (e.g. firewall). By consolidating the physical network devices, NFV provides greater flexibility for access networkby implementing network functions that can be moved to and/or instantiated in various locations in the networkwithout the installation and configuration of new physical network devices.

154 100 106 154 156 100 Network equipment performance databasemay store data collected by a performance management system. In access network, coverage areascover limited geographical areas. Performance databasemay store performance counters and events for these cells among other performance related information. These counters provide information about the performance and traffic load in specific cells, such as cell capacity, the amount of signaling in specific cells, etc. The network equipment configuration databasestores data that defines a configuration for access network.

104 130 158 100 1 FIG. A session, call, or data record is created for each UEusing messages, signals, and/or data collected or intercepted by monitoring probes from various network interfaces. A monitoring system, such as monitoring probes and monitoring server (not shown in), may be coupled to interfaces and links in the network to passively monitor and collect signaling data from one or more interfaces in the network. The monitoring system may collect user plane and control plane data from the interfaces. The monitoring probes, may comprise, for example, one or more processors running one or more software applications that collect, correlate and analyze Protocol Data Units (PDU) and data packets from both RAN and EPCnetwork interfaces and links. All collected data may be stored in network signaling database. Another example of signaling data collected from access networkwould be to obtain switching and routing table information or subscribers IP trace route information using well-known real time processing programs such as rtTrace.

159 3 100 130 Fault management databasepreferably stores information related to a service related network fault events. The events may be collected from a distributed network event database representing different subsystems (transport, layercontrol signaling, tunneled user plane, etc.) or protocol layers of access network. It is noted that fault management data can be aggregated on different levels, that is per cell, per event, per service, per area, etc., based on data obtained from RAN network elements and EPCelements. However, common communication systems have in fact limited the set of possible fault events, for the reason that it may be difficult to represent all possible problem sources. For example, routers typically report lost packets. However, they do not report packet reordering. Packet reordering as such is not considered to be an important performance fault in transport networks. Nevertheless, there are services that are sensitive to high levels of reordering of packets. These types of fault events, packet reordering in routers, are therefore not detectable by fault management systems that are purely based on fault reporting from the network.

1 FIG. 150 160 160 160 150 160 130 164 166 166 160 166 160 160 166 166 As shown in, troubleshooting systemmay be communicatively connected to a knowledge base system (KBS). In some embodiments, the KBSmay comprise a cloud-based system suited for complex tasks. According to an embodiment of the present invention, KBSmay be capable of providing resolution recommendations for a plurality of detected network failures by communicating with the troubleshooting system. In one embodiment, KBSmay be designed to apply formal representations of domain knowledge or expertise to solve network related problems. Symbolic descriptions (e.g., in the form of rules) of this expertise characterize the definitional and empirical relationships in a domain. This approach of knowledge models has been found advantageous in automating troubleshooting tasks that may become too complex to be accomplished by human experts. In an embodiment of the present invention, KBSmay comprise, or otherwise may cooperate with a rules engineand RAN analysis engine software program. RAN analysis enginemay comprise program instructions stored on one or more computer-readable storage devices, which may include internal storage on the knowledge base system. RAN analysis enginemay be, for example, a computer program or program component utilized as the inference engine of knowledge based systemthat matches the current inputs to relevant elements in knowledge base. In some embodiments, RAN analysis enginemay provide the means to re-assess the state of a situation during each cycle of a reasoning mechanism. As a result, RAN analysis enginemay be capable of reacting to a dynamic situation more readily than conventional programs.

160 162 162 168 Additionally, KBSmay include a user interface, such as graphical user interface (GUI)program. In one embodiment, GUImay comprise a framework for collecting user feedback from various users, such as network technicians, with respect to the actual causes of the reported network issues and with respect to recommended countermeasures. This historical information may be incorporated into the root cause analysis to provide more accurate recommendations as described in more detail below.

2 FIG. 2 FIG. 1 3 3 FIGS., andA-C 2 FIG. is a flowchart illustrating operation of an embodiment of the present invention. Before turning to description of, it is noted that the flow diagram shown therein is described, by way of example, with reference to steps and/or elements shown in, although these operational steps may be carried out in any system and are not limited to the scenario shown in the aforementioned figure. Additionally, the flow diagram inillustrates an example in which operational steps are carried out in a particular order, as indicated by the lines connecting the blocks, but the various steps shown in this diagram can be performed in any order, or in any combination or sub-combination. It should be appreciated that in some embodiments some of the steps described below may be combined into a single step. In some embodiments, one or more additional steps may be included.

202 166 168 162 202 168 According to an embodiment of the present invention, at step, the RAN analysis enginereceives RAN area of interest information and desired outcome information from the uservia GUI. In other words, at step, the userindicates a geographical area of interest where RAN troubleshooting or RAN optimization is needed. For example, RAN area of interest may be as large as a particular geographical market or as small as a particular group of adjacent cells. RAN troubleshooting and/or RAN optimization may be related to network-based issues such as, but not limited to, subscriber accessibility (e.g., attaching to an access network), retainability (e.g., call drops), or Quality of Service (QoS) (e.g., capacity/speed of a serving cell of the access network, voice/video MOS (Mean Opinion Score)), and the like. The desired outcome indicates the requested improvement. For example, the desired outcome may include, but is not limited to, improved cell coverage, improved call drop counts, improved data speed, and improved QoS metric or a combination thereof.

204 166 166 102 158 At step, the RAN analysis engineidentifies and retrieves values for a plurality of wireless network monitoring parameters related to the identified network areas for optimization. The identified plurality of parameters may include both RAN performance data (e.g., WWAN performance data) and core network performance data. In other words, the RAN analysis enginemay obtain ever-increasing 2g, 3g, 4g, LTE, or 5g variables related to user and/or control plane signaling from one of the network elements, such as the base station, or probe instrumentations that may be stored in network signaling database, for example. Examples of RAN variables may include NEM parameters and cell based counters and subscriber RRC data, such as radio bearer abnormal drop counters, radio frequency parameters such as RF frequency (channel), RF power (reference signal received power (RSRP)), RF interference (reference signal received quality (RSRQ)), signal-to-interference-plus-noise-ratio (SINR), RF RRC retransmissions and the like. Core network variables may include signaling related to subscriber sessions and communication flows associated with the subscribers, such as subscriber QoS attributes. Additionally, core network variables may include subscriber user plane signaling and QoS attributes such as uplink/downlink bandwidth, TCP retransmissions, voice/video MOS as a measure of user's quality of experience, application deep packet classification (e.g., Facebook application, Skype application, and the like).

206 166 166 150 166 166 At step, the RAN analysis enginecorrelates the network monitoring parameters to a particular cell or to a particular subscriber. In other words, the RAN analysis enginecorrelates big data that may include the identified RAN variables, core network variables, subscriber data and network data received from the troubleshooting system. At least in some embodiments, the RAN analysis enginemay reduce the list of correlated parameters to the ones that are most relevant for generating a recommendation. In various embodiments, the RAN analysis enginemay filter the amount of big data to be analyzed to certain categories: cell area of interest, critical variables required by the desired outcome, NEM cell-based counters, RAN network element tables (locations of cell sites), and the like.

208 166 Optionally, at step, the RAN analysis enginemay store the correlated network monitoring parameters in a data repository. This step may involve database insertion and indexing of data structures. In one embodiment, the data repository may comprise a relational database system. However, a major problem with relational database systems is that a good relationship schema requires many tables having columns in common between them. Accordingly, it is often necessary to calculate many joins to execute queries. In some embodiments, more efficient systems may be used as data repositories, such as storage systems based on HDFS (Hadoop Distributed File System) cloud storage systems, for example.

210 166 164 160 164 164 164 164 164 166 According to embodiment of the present invention, at step, the RAN analysis engineperforms root cause analysis of identified and/or stored network monitoring parameters using automated classification model by applying one or more knowledge processing rules. Broadly, knowledge representation is the activity of making abstract knowledge explicit, as concrete data structures, to support machine-based storage, management (e.g., information location and extraction), and reasoning systems. Knowledge processing rules may be applied using the rules engine software component, e.g., implemented by programming instructions encoded in one or more tangible, non-transitory computer-readable storage media executed by one or more processors of the KBSto provide the rules engine. According to embodiments of the present invention, these knowledge processing rules managed by the rules enginemay feed the machine learning based automated classification model. At least initial rule sets utilized by the rules enginemay include exemplary basic rules specifying typical RF power levels where drop calls start occurring and/or exemplary basic rules specifying RF interference levels where user's quality of experience starts dropping to unacceptable levels. As the automated classification model continually derives more information related to wireless network performance, the knowledge processing rules are selectively updated where relevant by the rules engine. The updated knowledge processing rules may be provided as real-time feedback by the rules engineto the RAN analysis engine.

166 In various embodiments, the RAN analysis engineutilizes one or more automated classification models that may utilize one of the automated machine learning techniques. Each model can cover a broad set of instances.

Conventional methods of incorporating machine learning, planning, and prediction have centered on three principal techniques. Statistical inference systems encompass not only traditional statistical analysis approaches (such as those supported by SAS and SPSS), but also machine learning capabilities based on clustering, Bayesian probabilities, decision trees, as well as partitioning and classification. Expert systems are if-then-else rule driven applications that apply an “inference engine” to evaluate rules and follow a path to a particular solution. The rules can be developed by subject matter experts or as part of a statistical analysis (in a manner similar to statistical interference systems) or through the process of data mining. The third technique is the use of neural networks, representing layered connectionist machines (either in software or hardware) that mimic the way neurons in the human brain supposedly collect data, aggregate related data elements, prioritize important data relationships, and learn patterns.

166 These approaches are not mutually exclusive. For example, rules can be generated by the statistical analysis of data patterns in a process known as “data mining.” In one embodiment, the RAN analysis enginemay utilize connectionlist and/or Bayesian machine-learning techniques to perform the root cause analysis. These automated techniques rely on statistical analysis and provide means of using the large population of variables that typically influence performance of a particular cell (e.g., coverage, capacity, or Quality of Service). The foregoing description presents an exemplary embodiment of the Bayesian techniques of the present invention. It will be understood, however, that the present invention is not limited to the particular embodiment described, and several variations and enhancements can be implemented as desired or necessitated by operational requirements.

212 166 210 166 166 212 166 212 166 166 3 3 FIGS.A-C According to an embodiment of the present invention, at step, the RAN analysis enginegenerates a recommendation related to the desired outcome based on the analysis performed at step. In various embodiments, the RAN analysis engineprovides meaningful predictive analysis of deployed radio cells, detect the existence of problem states within the wireless network, and/or perform root cause analysis. The RAN analysis enginecan preferably report problems that are either fully or partially “matched,” along with a confidence factor (e.g., confidence probability) associated with each matched problem. The machine-learning can advantageously leverage network data (e.g., the information associated with the plurality of variables that influence performance of a particular cell) that is compiled in the automated classification model. For example, at stepthe RAN analysis enginemay simply predict a root cause of a problem/issue with a wireless carrier network, such as dropped call or poor VoLTE MOS, based on the performed network data analysis. In another example, at step, the RAN analysis enginemay indicate a resolution/solution for the identified root cause of at least one service problem, such as, recommendation for additional cell tower or site or may recommend frequency change which improves cell coverage, for example. As another example, the generated recommendation may propose changes to the NEM configuration parameters, such as RF, MAC, and PHY parameters in such a way that minimizes the RF interference in the respective cluster of cells while maximizing the cell traffic capacity, as well as changing configuration of connections, links, and nodes. In other words, the recommendation may propose changes to radio frequency (RF) and physical layer (PHY) parameters of the cells (e.g., channel frequency, RF output power, segment). This may help to improve the intra-cell quality of service (QoS) by minimizing the intra-cell and inter-cell interference. As yet another non-limiting example, the generated recommendation could include recommendation to adjust antenna tilt and/or azimuth configuration and transmit power if the cell overlaps with different neighbor cells. The antenna tilt and transmit power level is used to communicate wireless signals in a cell. As noted above and as shown in, the RAN analysis enginemay also generate confidence factors associated with each recommendation. At least in some embodiments, the generated recommendation may also include, but is not limited to, the degree of efficacy of the input variables that influenced the recommendation (which may be based on the theoretical design of Taguchi, for example).

214 166 At step, the RAN analysis enginemay optionally automatically adjust configuration(s) of one or more RAN parameters based on the generated recommendation. Such parameters may also be expressed in terms of an input to NEM Operations or SON functions allowing automatic or semiautomatic reconfiguration of the RAN system.

166 According to embodiments of the present invention, the automated classification model is a Bayesian statistical model that utilizes sets of knowledge processing rules. Expert knowledge is used to seed training of the model by a machine. This knowledge-based seeding of the model may more effectively create a predictive model. In some embodiments, the seed may represent relationships between the plurality of network monitoring parameters. At least some of the rules may include one or more preconditions associated with certain confidence factors based on the knowledge-based seeding. The reliability of the derived recommendation depends on each confidence factor utilized during derivation process. These confidence factors are indicative of the probability of occurrence of a corresponding cause. According to an embodiment of the present invention, the automated classification model utilized by the RAN analysis engineis capable of improving credibility results for each derived recommendation as confidence factors can be automatically updated after each successful/unsuccessful prediction.

Table 1 below illustrates an exemplary rule set containing initial preset confidence factors (probabilities) for an exemplary cause code “radio coverage issue” associated with a particular problem of lost radio connection with a UE:

TABLE 1 P − P UE lost 0.95 0.05 Serving Cell −120 dBm < RSRP < −100 dBm 0.7 0.3 Serving Cell RSRP < −120 dBm 0.95 0.05 Serving Cell is “Outdoor Cell” 0.6 0.4 − 166 In table 1, P represents probability that a precondition is true and Prepresents probability that the precondition is false. The last row in table 1 represents probabilities assigned to a precondition that the serving cell is an outdoor cell, in case the NE table containing detailed information about each cell is not available to the RAN analysis engine.

3 FIG.A 3 FIG.A 3 FIG.A 3 FIG.A 300 304 306 308 310 312 314 166 304 308 312 300 166 166 166 166 304 314 166 depicts an example of a decision tree utilized by the automated classification model, in accordance with an embodiment of the present invention. More specifically, the decision treedepicted inrepresents the rule set (both pre-conditions and corresponding confidence factors) stored in the Table 1. In, nodesandrepresent true and false probabilities, respectively, for a particular cause, nodesandrepresent true and false probabilities, respectively, for serving cell RSRP values being between −120 dBm and −100 dBm, nodesandrepresent true and false probabilities, respectively, for a serving cell being an outdoor cell. According to an embodiment of the present invention, the RAN analysis enginemay determine a confidence factor that a root cause is an outdoor radio coverage issue by combining corresponding individual probabilities of preconditions,andusing the decision tree. Using the example in, the RAN analysis enginedetermines confidence factor of root cause being an outdoor coverage issue=P (cause)*P (RSRP value being between −120 dBm and −100 dBm)*P (outdoor serving cell)=0.95*0.7*0.6=0.399=39.9%. Similarly, the RAN analysis enginedetermines that probability of root cause being an indoor coverage issue=P (cause)*P (RSRP value being between −120 dBm and −100 dBm)*P (indoor serving cell)=0.95*0.7*0.4=0.266=26.6%. As yet another example, the RAN analysis enginedetermines that probability of root cause being an outdoor radio coverage issue, while RSRP measurement is outside of the range between −120 dBm and −100 dBm=P(cause)*P (RSRP value being outside of the −120 dBm and −100 dBm range)*P (outdoor serving cell)=0.95*0.3*0.6=0.171=17.1%. In a similar fashion, the RAN analysis enginemay determine confidence factors for each possible combination of nodes-in order to determine the most likely cause of the reported problem. In the illustrated case, the RAN analysis enginemay derive that the outdoor radio coverage issue is the most likely cause by comparing all computed confidence factors.

166 168 162 166 168 166 312 166 168 166 166 308 166 166 RSRP According to an embodiment of the present invention, the RAN analysis enginemay be configured to interact with the uservia the GUI. Continuing with the example of the outdoor radio coverage issue, the RAN analysis enginemay ask the userto confirm that the location of interest is an outdoor cell. If user's response is affirmative, the RAN analysis enginemay automatically update corresponding probability. In other words, P (outdoor serving cell)becomes 1. As another example, the RAN analysis enginemay ask the userhow sure he/she is that the root cause is a radio coverage issue. If the userenters 90%, the RAN analysis engineupdates Pwith probability value equal to 0.9. After the update, the RAN analysis enginemay recalculate the confidence factor of root cause being an outdoor coverage issue=P (cause)*P (RSRP value being between −120 dBm and −100 dBm)*P (outdoor serving cell)=0.95*0.9*1=0.855=85.5% As illustrated in this example, interaction with the usercan dramatically increase confidence factor of the predicted root cause of the problem.

Generally, all causes could be grouped into two different classifications—normal releases and abnormal releases, for example. Normal releases may include, but not limited to the following: “User inactivity,” “Load balancing tracking area update required,” “CS (circuit switched) fall back triggered,” “UE not available for PS (packet switched) service,” “Normal release,” “Detach,” “Successful handover,” “interRAT(Radio Access Technologies)Redirection,” and the like. Abnormal releases may include, but are not limited to the following: “Unspecified”, “Release due to E-UTRAN generated reason,” “Handover target not allowed,” “Cell not available,” “No radio resources available in target cell,” “Unknown or already allocated MME UE S IAP ID,” “Unknown or inconsistent pair of UE S IAP ID,” “Radio connection with UE lost,” “Load balancing TAU (Tracking Area Update) required,” “Failure in the radio interface procedure,” “Interaction with other procedure,” “Unknown E-RAB (E-UTRAN Radio Access Bearer) ID”, “encryption and/or integrity protection algorithms not supported,” “invalid CSG (Closed Subscriber Group) id,” “Transport resource unavailable,” “Control processing overload,” “Hardware failure,” and the like.

166 316 320 300 300 300 300 3 FIG.B 3 FIG.A 3 FIG.B It should be noted that each RAN parameter added to a particular decision tree doubles the number of decision points (nodes). According to an embodiment of the present invention, the RAN analysis enginemay utilize a plurality of decision trees, wherein each decision tree represents a rule set associated with a particular cause.depicts an example of the plurality of decision trees utilized by the automated classification model, in accordance with an embodiment of the present invention. Each of the two additional RAN parameters RSRQand distance from the serving celldouble the number of decision points in the decision treea, as compared to the decision treein. Furthermore,illustrates a plurality of decision treesa-d where each decision tree represents a rule set associated with a particular cause.

3 FIG.C 300 depicts yet another example of a decision tree that may be utilized by the automated classification model, in accordance with an embodiment of the present invention. The decision treee is associated with the measured throughput degradation. Generally, throughput degradation may be determined by comparing measured cell throughput value with a reference value, such as mean cell throughput of the entire network or mean cell throughput of a particular market or mean cell throughput of a particular coverage area.

Table 2 below illustrates an exemplary rule set containing initial preset confidence factors (probabilities) for an exemplary cause code “radio interference cell edge” associated with the throughput degradation problem:

TABLE 2 P − P Serving Cell RSRQ < −16 dBm 0.95 0.05 Neighbor Cell(s) RSRQ < −16 dBm 0.95 0.05 Distance from Serving cell (TA) > 800 m 0.8 0.2 Traffic Hotspot detected at location with distance from 0.99 0.01 serving cell > 800 m

300 332 334 336 338 340 342 344 346 3 FIG.C 3 FIG.C The decision treee depicted inrepresents the rule set (both pre-conditions and corresponding confidence factors) stored in the Table 2. In, nodesandrepresent true and false probabilities, respectively, for serving cell's RSRQ value being less than −16 dBm, nodesandrepresent true and false probabilities, respectively, for neighbor cell RSRP values being less than −16 dBm, nodesandrepresent true and false probabilities, respectively, for distance from serving cell being greater than 800 meters, and nodesandrepresent true and false probabilities, respectively, for traffic hotspot detected at location with distance from serving cell greater than 800 m.

166 300 332 336 340 344 166 166 166 212 166 3 FIG.C 2 FIG. S RSRQ N RSRQ TA Hotspot S RSRQ N RSRQ TA Hotspot − According to an embodiment of the present invention, the RAN analysis enginemay use the decision treee to determine confidence factor that the root cause of the throughput degradation is a concentration of subscribers that generate high demand for downlink traffic at serving cell edge by combining corresponding individual probabilities of preconditions,andand. Using the example in, the RAN analysis enginedetermines confidence factor of the root cause of the throughput degradation is a concentration of subscribers that generate high demand for downlink traffic at serving cell edge=P*P*P*P=0.95*0.95*0.8*0.99=0.715=71.5%. Similarly, the RAN analysis enginedetermines confidence factor that the root cause of the throughput degradation are few subscribers that generate high demand for downlink traffic while unexpectedly located at the serving cell edge=P*P*P*P=0.95*0.95*0.8*0.01=0.007=0.7%. In the illustrated case, the RAN analysis enginemay derive that the root cause of the throughput degradation is a concentration of subscribers that generate high demand for downlink traffic at serving cell edge by comparing all computed confidence factors. In this case, at step(shown in), the RAN analysis enginemay generate a recommendation to consider deployment of a small network cell in the detected hot spot area.

Advantageously, the various embodiments described herein provide automated techniques for troubleshooting failures in a telecommunication network by analyzing actual network and subscriber data. The aforementioned embodiments use knowledge based model containing rule sets that can significantly reduce troubleshoot and repair time. Furthermore, the automated system disclosed herein is preferably configured to provide confidence factors associated with each recommendation, which could substantially eliminate guesswork by network operators.

As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or computer program product. Accordingly, aspects of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “circuit,” “module” or “system.” Furthermore, aspects of the present invention may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied thereon.

Any combination of one or more computer readable medium(s) may be utilized. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the “C” programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).

Aspects of the present invention are described above with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

4 FIG. 150 160 150 160 150 160 illustrates internal and external components of the troubleshooting systemand KBSservers in accordance with an illustrative embodiment. The serversandare only examples of a suitable server computer and are not intended to suggest any limitation as to the scope of use or functionality of embodiments of the invention described herein. Regardless, the serversandare capable of being implemented and/or performing any of the functionality set forth hereinabove.

150 160 150 160 The serversandare operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well-known computing systems, environments, and/or configurations that may be suitable for use with the serversandinclude, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed data processing environments that include any of the above systems or devices, and the like.

150 160 150 160 The serversandmay be described in the general context of computer system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, and so on that perform particular tasks or implement particular abstract data types. The serversandmay be practiced in distributed data processing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed data processing environment, program modules may be located in both local and remote computer system storage media including memory storage devices.

150 160 150 160 416 428 418 428 416 4 FIG. The serversandare shown inin the form of a general-purpose computing device. The components of the serversandmay include, but are not limited to, one or more processors or processing units, a system memory, and a busthat couples various system components including the system memoryto the processor.

418 The busrepresents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. By way of example, and not limitation, such architectures include Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

150 160 150 160 The serversandtypically include a variety of computer system readable media. Such media may be any available media that is accessible by the serversand, and it includes both volatile and non-volatile media, removable and non-removable media.

428 430 432 150 160 434 418 428 The system memorycan include computer system readable media in the form of volatile memory, such as random access memory (RAM)and/or cache memory. The serversandmay further include other removable/non-removable, volatile/non-volatile computer system storage media. By way of example only, a storage systemcan be provided for reading from and writing to a non-removable, non-volatile magnetic media (not shown and typically called a “hard drive”). Although not shown, a magnetic disk drive for reading from and writing to a removable, non-volatile magnetic disk, and an optical disk drive for reading from or writing to a removable, non-volatile optical disk such as a CD-ROM, DVD-ROM or other optical media can be provided. In such instances, each can be connected to the busby one or more data media interfaces. As will be further depicted and described below, the memorymay include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.

440 415 162 164 166 428 415 A program/utility, having a set (at least one) of program modules, such as the GUI, rules engine, and RAN analysis engine, may be stored in the memoryby way of example, and not limitation, as well as an operating system, one or more application programs, other program modules, and program data. Each of the operating system, one or more application programs, other program modules, and program data or some combination thereof, may include an implementation of a networking environment. Program modulesgenerally carry out the functions and/or methodologies of embodiments of the invention as described herein.

150 160 414 424 150 160 150 160 422 150 160 420 420 150 160 418 150 160 The serversandmay also communicate with one or more external devicessuch as a keyboard, a pointing device, a display, etc.; one or more devices that enable a user to interact with the serversand; and/or any devices (e.g., network card, modem, etc.) that enable the servers,to communicate with one or more other computing devices. Such communication can occur via Input/Output (I/O) interfaces. Still yet, the servers,can communicate with one or more networks such as a local area network (LAN), a general wide area network (WAN), and/or a public network (e.g., the Internet) via a network adapter. As depicted, the network adaptercommunicates with the other components of the servers,via the bus. It should be understood that although not shown, other hardware and/or software components could be used in conjunction with the servers,. Examples, include, but are not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The descriptions of the various embodiments of the present invention have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application or technical improvement over technologies found in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.