Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A reinforcement learning-based resource allocation method for a wireless backhaul network, the method comprising: estimating, by a resource allocation apparatus, locations of a plurality of base stations based on channel state information (CSI) measured by the plurality of base stations; and allocating, by the resource allocation apparatus, resources of the wireless backhaul network to the plurality of base stations using a reinforcement learning neural network having the locations as an input, wherein the resource allocation apparatus estimates the locations by inputting, to a first neural network, first CSIs measured by the plurality of base stations using a reference signal transmitted by the wireless backhaul network and a second CSI measured with respect to adjacent neighboring base stations among the plurality of base stations.
This technical summary describes a reinforcement learning-based resource allocation method for wireless backhaul networks. The method addresses the challenge of efficiently allocating network resources to multiple base stations in a dynamic wireless environment. The system uses a resource allocation apparatus that first estimates the locations of base stations by analyzing channel state information (CSI). The apparatus inputs first CSIs, measured by the base stations using a reference signal from the backhaul network, and second CSIs, measured between adjacent neighboring base stations, into a first neural network. This neural network processes the input data to determine the base station locations. The apparatus then allocates resources to the base stations using a reinforcement learning neural network, which takes the estimated locations as input. The reinforcement learning model dynamically adjusts resource allocation to optimize network performance. The method improves efficiency and reliability in wireless backhaul networks by leveraging machine learning to adapt to changing conditions and optimize resource distribution.
2. The reinforcement learning-based resource allocation method of claim 1 , wherein the first neural network is a recurrent neural network (RNN).
This invention relates to reinforcement learning-based resource allocation systems, specifically addressing the challenge of efficiently distributing computational or network resources in dynamic environments. The method uses a reinforcement learning framework to optimize resource allocation decisions, improving performance and reducing waste. A key aspect is the use of a first neural network to process input data, such as resource demands and system states, and generate allocation decisions. The neural network is trained through reinforcement learning, where it receives feedback on the outcomes of its decisions and adjusts its parameters to maximize long-term rewards. The invention further specifies that the first neural network is a recurrent neural network (RNN), which is particularly suited for handling sequential data and temporal dependencies in resource allocation tasks. The RNN's ability to maintain state information over time allows it to make more informed decisions based on historical context. The method may also include a second neural network for additional processing, such as predicting future resource demands or refining allocation strategies. The overall system aims to dynamically adapt to changing conditions, ensuring optimal resource utilization while minimizing overhead. This approach is applicable in various domains, including cloud computing, network management, and distributed systems, where efficient resource allocation is critical for performance and cost-effectiveness.
3. The reinforcement learning-based resource allocation method of claim 1 , wherein the first CSIs are measured using reference signals transmitted by some antennas of the wireless backhaul network.
This invention relates to a reinforcement learning-based resource allocation method for wireless backhaul networks, addressing the challenge of efficiently allocating network resources to optimize performance. The method involves measuring first channel state information (CSIs) using reference signals transmitted by some antennas in the wireless backhaul network. These CSIs are used to train a reinforcement learning model, which then determines an optimal resource allocation strategy. The method also involves measuring second CSIs using reference signals transmitted by other antennas in the network, which are used to update the reinforcement learning model. The reinforcement learning model is trained to maximize a reward function, which is based on metrics such as throughput, latency, or energy efficiency. The method dynamically adjusts resource allocation in response to changing network conditions, improving overall network performance. The reinforcement learning model may be implemented using deep learning techniques, such as deep Q-networks or policy gradient methods, to handle complex and high-dimensional state spaces. The method is particularly useful in scenarios where network conditions are dynamic and unpredictable, such as in dense urban environments or high-mobility scenarios. By leveraging reinforcement learning, the method adapts to changing conditions in real-time, ensuring efficient and effective resource allocation.
4. The reinforcement learning-based resource allocation method of claim 1 , wherein the reinforcement learning neural network defines, as a state, a location and a required communication capacity of each of the plurality of base stations.
This invention relates to a reinforcement learning-based resource allocation method for optimizing communication capacity in wireless networks. The method addresses the challenge of efficiently allocating resources in dynamic environments where multiple base stations must adapt to varying communication demands. The core innovation involves using a reinforcement learning neural network to model and optimize resource distribution. The neural network defines the system state by capturing key parameters, including the physical location and required communication capacity of each base station in the network. By continuously monitoring these states, the network learns to allocate resources—such as bandwidth, power, or frequency channels—in real-time to meet demand while minimizing waste. The reinforcement learning approach allows the system to adapt to changing conditions, such as user mobility or traffic fluctuations, without manual intervention. The method improves upon traditional static allocation schemes by dynamically adjusting resources based on learned patterns and feedback. This ensures higher efficiency, reduced interference, and better overall network performance. The neural network’s ability to process and respond to real-time data makes it particularly suitable for modern wireless systems, including 5G and beyond, where demand is highly variable. The invention enhances scalability and reliability in large-scale deployments by automating resource management.
5. The reinforcement learning-based resource allocation method of claim 1 , wherein the reinforcement learning neural network has an action of allocating, to at least one of the plurality of base stations, a plurality of narrow beams and allocable resources supported by an ultra-wide-area wireless backhaul network.
This invention relates to a reinforcement learning-based method for resource allocation in wireless communication networks, specifically addressing the challenge of efficiently distributing narrow beams and network resources across multiple base stations in an ultra-wide-area wireless backhaul network. The method leverages a reinforcement learning neural network to dynamically allocate resources, optimizing performance in dense or high-traffic environments where traditional static allocation schemes are inefficient. The neural network learns from network conditions and user demand patterns to determine the optimal allocation of narrow beams—highly directional, focused signals—to specific base stations, ensuring minimal interference and maximizing coverage. Additionally, the system allocates other network resources, such as bandwidth and computational capacity, supported by the backhaul network, which connects base stations to the core network. The reinforcement learning approach allows the system to adapt in real-time, improving spectral efficiency and reducing latency. This method is particularly useful in scenarios requiring dynamic adjustments, such as during peak usage times or when network conditions change rapidly. The neural network's action space includes both beam allocation and resource distribution, enabling a holistic optimization strategy. By continuously learning from feedback, the system enhances network reliability and user experience in large-scale wireless deployments.
6. The reinforcement learning-based resource allocation method of claim 1 , wherein the reinforcement learning neural network includes a neural network for determining a Q value in Q-learning.
This invention relates to reinforcement learning-based resource allocation systems, specifically addressing the challenge of efficiently allocating resources in dynamic environments where optimal decisions must be made under uncertainty. The method employs a reinforcement learning neural network to optimize resource distribution, improving performance in applications such as cloud computing, network management, or task scheduling. The neural network is designed to determine a Q value in Q-learning, a reinforcement learning algorithm that estimates the expected future rewards of actions to guide decision-making. The Q value represents the quality of a particular action in a given state, helping the system select actions that maximize long-term rewards. By integrating this Q-learning mechanism, the method enables adaptive and data-driven resource allocation, reducing inefficiencies and improving system responsiveness. The neural network processes input data representing the current state of the system, such as resource availability, demand, or environmental conditions, and outputs an optimal allocation strategy. The Q-learning component continuously updates its value estimates based on feedback from the environment, allowing the system to learn and refine its decisions over time. This approach ensures that resource allocation remains efficient even as conditions change, enhancing overall system performance. The method is particularly useful in scenarios where traditional rule-based or static allocation strategies fail to adapt to dynamic demands.
7. The reinforcement learning-based resource allocation method of claim 1 , wherein the reinforcement learning neural network determines a reward on the basis of at least one of an average capacity of the entire network, an average interference level measured by the plurality of base stations, an average capacity of some users who have received services from the plurality of base stations, and total energy used for communication.
This invention relates to a reinforcement learning-based method for optimizing resource allocation in wireless communication networks. The method addresses the challenge of efficiently distributing network resources, such as bandwidth and power, to maximize performance while minimizing interference and energy consumption. The system uses a reinforcement learning neural network to dynamically allocate resources across multiple base stations. The neural network evaluates network conditions and adjusts resource allocation in real-time to improve overall efficiency. Key performance metrics include network capacity, interference levels, user service quality, and energy usage. The neural network determines a reward function based on multiple factors, including the average capacity of the entire network, the average interference measured by the base stations, the average capacity of specific users receiving services, and the total energy consumed for communication. By optimizing these metrics, the method aims to enhance network performance, reduce interference, and lower energy consumption. The reinforcement learning approach allows the system to adapt to changing network conditions, such as varying user demand or environmental factors, without requiring manual intervention. This dynamic optimization ensures that resources are allocated in a way that balances efficiency, quality of service, and sustainability. The method is particularly useful in dense wireless networks where traditional static allocation strategies may lead to inefficiencies or degraded performance.
9. A reinforcement learning-based resource allocation apparatus for a wireless backhaul network, the reinforcement learning-based resource allocation apparatus comprising: a communication apparatus configured to receive channel state information (CSI) measured by a plurality of base stations; a storage configured to store a first neural network for estimating locations of the plurality of base stations using a plurality of pieces of CSI and a second neural network for determining allocation of resources to the plurality of base stations using reinforcement learning on the basis of the estimated locations; and a processor configured to estimate the locations of the plurality of base stations by inputting the CSI to the first neural network and determine allocation of resources of the wireless backhaul network to the plurality of base stations on the basis of a reward for a current state determined by inputting the estimated locations to the second neural network, wherein the CSI includes first CSIs measured by the plurality of base stations using a reference signal transmitted from the wireless backhaul network and a second CSI measured with respect to adjacent neighboring base stations among the plurality of base stations.
This invention relates to a reinforcement learning-based resource allocation system for wireless backhaul networks, addressing the challenge of efficiently allocating network resources in dynamic wireless environments. The system includes a communication module that collects channel state information (CSI) from multiple base stations, which includes both CSI measured from a reference signal transmitted by the backhaul network and CSI measured between adjacent base stations. A storage unit holds two neural networks: the first estimates base station locations using the collected CSI, while the second uses reinforcement learning to allocate resources based on these estimated locations. A processor inputs the CSI into the first neural network to determine base station positions, then uses the second neural network to optimize resource allocation by evaluating rewards for different network states. The reinforcement learning approach dynamically adjusts resource distribution to improve network performance, considering both the backhaul network's reference signal and inter-base station interference. This system enhances backhaul network efficiency by leveraging machine learning to adapt resource allocation in real-time.
10. The reinforcement learning-based resource allocation apparatus of claim 9 , wherein the first neural network is a recurrent neural network (RNN).
A reinforcement learning-based resource allocation system improves the efficiency of resource distribution in dynamic environments, such as cloud computing or network management, by optimizing allocation decisions in real-time. Traditional resource allocation methods often rely on static rules or heuristic approaches, which struggle to adapt to changing conditions and may lead to suboptimal performance. This system addresses the problem by using a reinforcement learning framework to dynamically adjust resource allocation based on feedback from the environment. The system includes a first neural network, which is a recurrent neural network (RNN), to process sequential data and capture temporal dependencies in resource usage patterns. The RNN's ability to maintain state information over time allows it to make more informed allocation decisions compared to feedforward networks. The system also incorporates a second neural network, which may be a convolutional neural network (CNN), to extract spatial features from resource usage data, such as network traffic patterns or workload distributions. These neural networks work together to generate allocation policies that maximize resource utilization while minimizing waste or bottlenecks. The reinforcement learning framework enables the system to learn optimal allocation strategies through trial and error, using rewards or penalties based on performance metrics like latency, throughput, or energy efficiency. The system continuously updates its policies to adapt to changing conditions, ensuring sustained performance in dynamic environments. This approach provides a more flexible and adaptive solution compared to traditional methods, improving overall system efficiency and responsiveness.
11. The reinforcement learning-based resource allocation apparatus of 9 , wherein the first CSIs are measured using reference signals transmitted by some antennas of the wireless backhaul network.
This invention relates to a reinforcement learning-based resource allocation system for wireless backhaul networks, addressing the challenge of efficiently allocating network resources to optimize performance. The system uses channel state information (CSI) measured from reference signals transmitted by some antennas in the network to train a reinforcement learning model. The model dynamically allocates resources, such as bandwidth or power, to improve data transmission efficiency and reduce interference. The apparatus includes a measurement module that collects CSI from the reference signals, a reinforcement learning module that processes this data to make allocation decisions, and an allocation module that implements the decisions across the network. The system adapts to changing network conditions by continuously updating the reinforcement learning model with new CSI measurements, ensuring optimal resource distribution over time. This approach enhances network reliability and throughput while minimizing resource waste. The invention is particularly useful in dense wireless networks where traditional static allocation methods are inefficient.
12. The reinforcement learning-based resource allocation apparatus of claim 9 , wherein a state used in the reinforcement learning is defined using a location and a required communication capacity of each of the plurality of base stations.
A reinforcement learning-based resource allocation system optimizes network resource distribution in wireless communication environments. The system addresses the challenge of efficiently allocating limited network resources, such as bandwidth and computational power, across multiple base stations to meet varying communication demands while minimizing congestion and maximizing performance. The apparatus employs reinforcement learning to dynamically adjust resource allocation based on real-time network conditions. A key feature is the definition of the state used in the reinforcement learning process, which incorporates both the geographical location and the required communication capacity of each base station. By considering these factors, the system can make informed decisions about resource distribution, ensuring that high-demand areas receive adequate resources while avoiding over-allocation in low-demand regions. The reinforcement learning model continuously updates its policy based on feedback from the network, allowing it to adapt to changing conditions such as user mobility, traffic patterns, and environmental interference. This adaptive approach improves overall network efficiency, reduces latency, and enhances user experience by dynamically balancing resource allocation across the network infrastructure. The system is particularly useful in dense urban environments or high-traffic scenarios where traditional static allocation methods are insufficient.
13. The reinforcement learning-based resource allocation apparatus of claim 9 , wherein the processor determines the reward on the basis of at least one of an average capacity of the entire network, an average interference level measured by the plurality of base stations, an average capacity of some users who have received services from the plurality of base stations, and total energy used for communication.
This invention relates to a reinforcement learning-based resource allocation system for optimizing network performance in wireless communication environments. The system addresses the challenge of efficiently allocating resources such as bandwidth, power, and frequency channels across multiple base stations to maximize network capacity while minimizing interference and energy consumption. The apparatus includes a processor that dynamically adjusts resource allocation using reinforcement learning techniques. The processor evaluates the effectiveness of different allocation strategies by calculating a reward function based on key performance metrics. These metrics include the average capacity of the entire network, the average interference levels measured by the base stations, the average capacity of specific users served by the base stations, and the total energy consumed for communication. By continuously monitoring these factors, the system learns and adapts to optimize resource distribution in real time. The reinforcement learning model processes feedback from the network to refine its allocation decisions, ensuring that resources are allocated in a way that balances network efficiency, user experience, and energy efficiency. This approach enables the system to handle dynamic network conditions, such as varying user demand and environmental interference, while maintaining high performance and reliability. The invention is particularly useful in dense wireless networks where traditional static allocation methods are inefficient.
14. The reinforcement learning-based resource allocation apparatus of claim 9 , wherein the second neural network includes a neural network for determining a Q value in Q-learning.
The invention relates to a reinforcement learning-based resource allocation system designed to optimize the distribution of computational or network resources in dynamic environments. The system addresses the challenge of efficiently allocating resources in real-time to maximize performance while minimizing waste, particularly in scenarios where resource demands fluctuate unpredictably. The apparatus includes a first neural network that processes input data representing the current state of the system, such as available resources and demand patterns. This network generates an action vector that suggests how resources should be allocated. A second neural network evaluates the potential outcomes of these actions by calculating a Q value, a metric used in Q-learning to estimate the long-term reward of taking a specific action in a given state. The Q value helps the system select the most advantageous allocation strategy. The system also incorporates a reward function that quantifies the effectiveness of the resource allocation decisions, providing feedback to refine the neural networks over time. By continuously learning from this feedback, the apparatus adapts to changing conditions, improving efficiency and performance. The use of reinforcement learning allows the system to handle complex, multi-variable optimization problems that traditional rule-based methods struggle with. This approach is particularly useful in cloud computing, data centers, and network management, where resource allocation directly impacts system efficiency and cost.
15. The reinforcement learning-based resource allocation apparatus of claim 9 , wherein the processor allocates, to at least one of the plurality of base stations, a plurality of narrow beams and allocable resources supported by an ultra-wide-area wireless backhaul network on the basis of the reward for the current state and a reward for a previous state thereof.
This invention relates to a reinforcement learning-based resource allocation system for wireless networks, specifically addressing the challenge of efficiently distributing narrow beams and network resources across multiple base stations in an ultra-wide-area wireless backhaul network. The system uses reinforcement learning to optimize resource allocation by evaluating rewards associated with current and previous states of the network. The processor dynamically assigns narrow beams and allocable resources to base stations based on these rewards, improving network performance and resource utilization. The reinforcement learning model continuously adapts to changing network conditions, ensuring optimal allocation decisions. This approach enhances spectral efficiency, reduces interference, and maximizes throughput in large-scale wireless backhaul networks. The system is particularly useful in environments where traditional static allocation methods fail to keep up with dynamic traffic demands and environmental changes. By leveraging historical and current state rewards, the apparatus ensures that resource allocation remains efficient and responsive to real-time network conditions.
16. A machine learning-based resource allocation method for a wireless backhaul network, the machine learning-based resource allocation method being performed by a resource allocation apparatus and comprising: estimating locations of a plurality of base stations using a first neural network having channel state information (CSI) measured by the plurality of base stations as an input; and allocating resources of the wireless backhaul network to the plurality of base stations using a second neural network having the locations as an input, wherein the resource allocation apparatus estimates the locations of the plurality of base stations by inputting, to the first neural network, first CSIs measured by the plurality of base stations on the basis of reference signals transmitted from some antennas of the wireless backhaul network and a second CSI measured with respect to adjacent neighboring base stations among the plurality of base stations.
This technical summary describes a machine learning-based resource allocation system for wireless backhaul networks. The system addresses the challenge of efficiently allocating network resources in dynamic wireless backhaul environments where base station locations and channel conditions frequently change. The solution uses two neural networks to optimize resource allocation. The first neural network estimates the locations of multiple base stations by analyzing channel state information (CSI) derived from reference signals transmitted by certain antennas in the network, as well as CSI measured between adjacent base stations. The second neural network then uses these estimated locations to allocate network resources, such as bandwidth or transmission power, to the base stations. By leveraging machine learning, the system dynamically adapts to changing network conditions, improving efficiency and performance in wireless backhaul communications. The approach eliminates the need for manual configuration or static resource allocation, enhancing scalability and reliability in dense or heterogeneous network deployments.
17. The machine learning-based resource allocation method of claim 16 , wherein the second neural network is a Deep Q-Network (DQN) including a neural network for determining a Q value in Q-learning.
This invention relates to machine learning-based resource allocation systems, specifically for optimizing resource distribution in dynamic environments. The method addresses the challenge of efficiently allocating limited resources, such as computational power, bandwidth, or memory, in real-time systems where demand fluctuates unpredictably. Traditional resource allocation methods often rely on static rules or simple heuristics, which fail to adapt to changing conditions, leading to inefficiencies or resource waste. The invention uses a machine learning model to dynamically allocate resources based on current system demands and constraints. A first neural network predicts future resource requirements by analyzing historical data and real-time inputs. A second neural network, specifically a Deep Q-Network (DQN), determines optimal allocation decisions using Q-learning. The DQN evaluates potential actions (resource allocations) by estimating their long-term rewards, allowing the system to make decisions that maximize efficiency over time. The neural network for Q-learning processes state inputs, such as current resource usage and demand patterns, and outputs Q-values representing the expected future rewards of different allocation strategies. This enables the system to select the most advantageous allocation dynamically. The method improves upon prior approaches by incorporating reinforcement learning to adapt to evolving conditions, ensuring resources are allocated in a way that balances immediate needs with long-term system performance. This is particularly useful in environments like cloud computing, data centers, or IoT networks where resource demands vary significantly.
Unknown
October 13, 2020
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.