Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A computer implemented method for assigning executable jobs to pipeline sets comprising: generating a weight for each pipeline set of a plurality of pipeline sets to obtain a plurality of weights, wherein generating the weight for each pipeline set comprises: obtaining a plurality of duty cycle metrics for a plurality of pipeline threads in the pipeline set, wherein the plurality of duty cycle metrics comprises a measure of an amount of time that a corresponding pipeline thread is executing and actively processing data, and determining the weight for the pipeline set based at least in part on the plurality of duty cycle metrics; and assigning a job request to a target pipeline set selected from the plurality of pipeline sets according to a weighted random algorithm, wherein the weighted random algorithm uses the plurality of weights.
This invention relates to computer-implemented job scheduling and addresses the problem of efficiently assigning executable jobs to a group of processing pipelines. The method involves a computer system that first determines the suitability of each available pipeline set for receiving jobs. This is achieved by calculating a "weight" for each pipeline set. To calculate this weight, the system examines the performance of individual threads within that pipeline set. Specifically, it measures the "duty cycle" of each pipeline thread, which represents the amount of time a thread is actively executing and processing data. The weight for the entire pipeline set is then determined based on these duty cycle metrics. Once weights are generated for all pipeline sets, a new job request is assigned to a specific pipeline set. This assignment is made using a weighted random algorithm. This algorithm leverages the calculated weights, meaning pipeline sets with higher weights (indicating greater current activity or processing capacity) have a proportionally higher probability of being selected to receive the new job request. This ensures that jobs are distributed in a manner that considers the current operational status of the pipelines.
2. The computer implemented method of claim 1 , wherein generating the weight further comprises: determining an average number of data ingestion tasks assigned to the plurality of pipeline sets, wherein determining the weight is based on the average number of data ingestion tasks.
The invention relates to optimizing data processing in a distributed computing environment, specifically improving the allocation of data ingestion tasks across multiple pipeline sets. The problem addressed is inefficient task distribution, which can lead to bottlenecks, uneven workloads, or suboptimal resource utilization in data processing systems. The method involves dynamically generating a weight for each pipeline set based on the average number of data ingestion tasks assigned to them. This weight is used to balance the workload across the pipeline sets, ensuring that tasks are distributed more evenly and resources are utilized efficiently. The weight calculation considers historical or real-time data on task assignments, allowing the system to adapt to changing workloads or system conditions. The method may also include steps such as monitoring task assignments, analyzing performance metrics, and adjusting the weight in response to detected imbalances or inefficiencies. By incorporating the average number of tasks into the weight determination, the system can prioritize pipeline sets that are underutilized or deprioritize those that are overloaded, leading to smoother and more predictable data processing operations. This approach is particularly useful in large-scale data processing environments where task distribution must be optimized for performance and scalability.
3. The computer implemented method of claim 1 , wherein obtaining a duty cycle metric comprises: assigning a duty cycle ownership object to a thread, when a set of conditions is satisfied, setting the duty cycle ownership object to an on state, and processing a data object in a consumer queue while the duty cycle ownership object is in the on state, wherein the duty cycle ownership object is switched to an off state after processing the data object, wherein the set of conditions comprising the data object existing in the consumer queue of the thread, and keeping the duty cycle ownership object in the off state when the consumer queue is empty.
This describes a computer-implemented method for assigning executable jobs to pipeline sets. The method determines a weight for each pipeline set based on the processing activity of its threads and uses these weights to assign jobs via a weighted random algorithm. A core part of this weighting process is obtaining a "duty cycle metric" for each pipeline thread, which measures how much time a thread is actively executing and processing data. This is achieved by: 1. Assigning a "duty cycle ownership object" to a thread. 2. Setting this object to an "on" state when a data object appears in the thread's consumer queue. 3. Processing the data object while the ownership object is "on." 4. Switching the ownership object to an "off" state after the data object is processed. 5. Keeping the ownership object "off" when the consumer queue is empty, indicating the thread is idle. This on/off state tracking enables the system to calculate the thread's active processing time for its duty cycle metric. ERROR (embedding): Error: Failed to save embedding: Could not find the 'embedding' column of 'patent_claims' in the schema cache
4. The computer implemented method of claim 1 , wherein obtaining a duty cycle metric comprises: assigning a duty cycle ownership object to a thread, and when a set of conditions is satisfied, setting the duty cycle ownership object to an on state within a scope defined by determining that the set of conditions are satisfied, processing a data object in a consumer queue while within the scope, and exiting the scope to switch the duty cycle ownership object to an off state, wherein the set of conditions comprising the data object existing in the consumer queue of the thread.
This invention relates to computer-implemented methods for managing duty cycle metrics in multi-threaded systems, particularly for optimizing resource utilization and workload distribution. The problem addressed is inefficient processing in multi-threaded environments where threads may compete for shared resources, leading to contention and suboptimal performance. The method involves assigning a duty cycle ownership object to a thread, which controls access to a consumer queue. When specific conditions are met, the duty cycle ownership object transitions to an "on" state, enabling the thread to process data objects from the queue. The conditions include the presence of a data object in the thread's consumer queue. While the ownership object is in the "on" state, the thread processes the data object within a defined scope. Once processing is complete, the scope is exited, and the ownership object switches to an "off" state, releasing control. This mechanism ensures orderly access to shared resources, reducing contention and improving efficiency in multi-threaded processing environments. The approach dynamically adjusts based on workload availability, ensuring threads only process data when conditions are favorable, thus optimizing system performance.
5. The computer implemented method of claim 1 , wherein obtaining a duty cycle metric comprises: reading a plurality of timestamps and state information from a duty cycle ownership object, and generating, as a duty cycle metric, a weighted moving average using the plurality of timestamps and the state information.
This invention relates to monitoring and analyzing duty cycle metrics in computing systems, particularly for tracking resource utilization and ownership over time. The problem addressed is the need for accurate, real-time measurement of how computational resources are allocated and utilized across different processes or components, which is critical for performance optimization, load balancing, and fault detection. The method involves obtaining a duty cycle metric by reading timestamps and state information from a duty cycle ownership object. This object records when and how long a resource (e.g., a CPU, memory, or network bandwidth) is active or idle, along with the state of the system during those periods. The timestamps indicate when transitions between states occur, while the state information describes the operational mode (e.g., active, idle, or error) at those times. To generate the duty cycle metric, a weighted moving average is computed using the timestamps and state information. This approach provides a dynamic, time-sensitive measure of resource usage, where recent data points have greater influence on the metric than older ones. The weighted moving average smooths out short-term fluctuations while still reflecting current trends, making it useful for real-time monitoring and adaptive system management. The method ensures accurate tracking of resource allocation by continuously updating the duty cycle metric as new timestamps and state data are recorded. This allows for precise identification of bottlenecks, inefficient usage patterns, or potential failures, enabling proactive system adjustments. The technique is applicable in various computing environments, including cloud platforms, distributed systems, and embedded devices, where efficient resource mana
6. The computer implemented method of claim 1 , wherein generating the weight comprises: set, for at least a subset of the plurality of pipeline sets, a pipeline set duty cycle metric as a maximal duty cycle metric of the plurality of pipeline threads in the pipeline set, generating a sliding window average of incoming jobs to the plurality of pipeline sets, and using the sliding window average and the thread set duty cycle metric to determine the weight for the pipeline set.
This invention relates to optimizing job scheduling in a multi-threaded pipeline processing system. The problem addressed is efficiently distributing workloads across multiple pipeline sets, each containing multiple threads, to maximize throughput and minimize latency. The method involves dynamically assigning weights to pipeline sets based on their operational efficiency and current workload. The process begins by evaluating the duty cycle of each thread within a pipeline set, which measures the proportion of time the thread is actively processing jobs. For each pipeline set, the maximum duty cycle among its threads is selected as the pipeline set's duty cycle metric. This metric indicates the set's overall utilization and capacity. Next, a sliding window average of incoming jobs is calculated for each pipeline set, providing a real-time measure of workload demand. The sliding window average and the pipeline set duty cycle metric are then used to compute a weight for the pipeline set. This weight determines the priority or share of incoming jobs allocated to the set, ensuring that highly utilized sets receive fewer jobs while underutilized sets receive more, thereby balancing the workload and improving system efficiency. The method dynamically adjusts weights in response to changing workload patterns, optimizing resource allocation and preventing bottlenecks in the pipeline processing system.
7. The computer implemented method of claim 1 , wherein generating the weight comprises: determine, for the pipeline set, a pipeline set duty cycle metric from the plurality of duty cycle metrics, generating a sliding window average of incoming jobs to the plurality of pipeline sets, and when the sliding window average is equal to zero, calculating a sum of the pipeline set duty cycle metric across the plurality of pipeline sets, and determining an assigned value for the pipeline set as a difference between the sum and the pipeline set duty cycle metric, normalizing the assigned value across the plurality of pipeline sets to obtain the plurality of weights.
This invention relates to optimizing job scheduling in a computing system with multiple pipeline sets. The problem addressed is efficiently distributing workloads across pipelines to balance resource utilization and minimize idle time. The method involves generating weights for each pipeline set to prioritize job allocation based on their current and historical workload metrics. The process begins by determining a duty cycle metric for each pipeline set, representing its utilization over time. A sliding window average of incoming jobs is then calculated for each pipeline set. If the sliding window average is zero, indicating no recent job activity, the method calculates the sum of all pipeline set duty cycle metrics. The assigned value for a pipeline set is derived by subtracting its individual duty cycle metric from this sum. This value is then normalized across all pipeline sets to produce a set of weights. These weights are used to allocate incoming jobs, ensuring fair distribution and preventing overloading of any single pipeline set. The approach dynamically adjusts to workload changes, improving system efficiency and resource utilization.
8. The computer implemented method of claim 1 , wherein determining the weight for the pipeline set comprises: determine, for the pipeline set, a pipeline set duty cycle metric from the plurality of duty cycle metrics, generating a sliding window average of incoming jobs to the plurality of pipeline sets, and when the sliding window average is greater than zero, assigning a value to the pipeline set using a periodic update model to obtain an assigned value, and normalizing the assigned value across the plurality of pipeline sets to obtain the plurality of weights.
This invention relates to optimizing job scheduling in a computing system with multiple pipeline sets. The problem addressed is efficiently distributing incoming jobs across pipelines to balance workload and improve system performance. The method determines a weight for each pipeline set by analyzing duty cycle metrics and job inflow patterns. For each pipeline set, a duty cycle metric is calculated from the plurality of duty cycle metrics associated with the system. A sliding window average of incoming jobs is generated to assess recent job distribution trends. If the sliding window average exceeds zero, a periodic update model assigns a value to the pipeline set, which is then normalized across all pipeline sets to produce a weight. This weight is used to prioritize job allocation, ensuring fair and efficient resource utilization. The periodic update model dynamically adjusts weights based on real-time job inflow, while normalization ensures consistent scaling across all pipeline sets. The method improves load balancing by dynamically adapting to varying job loads and pipeline performance, reducing bottlenecks and enhancing overall system throughput.
9. The computer implemented method of claim 1 , wherein determining the weight for the pipeline set comprises: sorting the plurality of pipeline sets in increasing order of a plurality of pipeline set duty cycle metrics to create an ordered list, wherein each pipeline set duty cycle metric is determined from the plurality of duty cycle metrics for a corresponding pipeline set, generating a sliding window average of incoming jobs to the plurality of pipeline sets, and identifying a maximal position in the ordered list in which the sum of a pipeline set duty cycle metric at the maximal position minus a duty cycle metric of a subset of the plurality of duty cycle metrics up to the maximal position is greater than or equal to the sliding window average, the pipeline set duty cycle metric being in the plurality of pipeline set duty cycle metrics, and setting an assigned value to zero for each pipeline set in the ordered list that is at a position greater than the maximal position.
The invention relates to optimizing job scheduling in a computing system with multiple processing pipelines. The problem addressed is efficiently distributing incoming jobs across pipeline sets to balance workload and prevent bottlenecks. The method involves analyzing pipeline performance metrics to dynamically assign weights to pipeline sets, ensuring fair and efficient job distribution. The process begins by sorting pipeline sets based on their duty cycle metrics, which measure utilization levels. These metrics are aggregated for each pipeline set to create an ordered list from lowest to highest utilization. A sliding window average of incoming jobs is then calculated to estimate current workload trends. The method identifies a critical point (maximal position) in the ordered list where the cumulative duty cycle metric up to that point, minus the individual duty cycle metric at the maximal position, meets or exceeds the sliding window average. Pipeline sets beyond this position are assigned a weight of zero, effectively excluding them from receiving new jobs. This ensures that underutilized pipelines receive more jobs, while overloaded ones are temporarily bypassed, improving overall system efficiency. The approach dynamically adjusts to workload fluctuations, maintaining balanced pipeline utilization.
10. The computer implemented method of claim 1 , wherein determining the weight for the pipeline set comprises: sorting the plurality of pipeline sets in increasing order of a plurality of pipeline set duty cycle metrics to create an ordered list, wherein each pipeline set duty cycle metric is determined from the plurality of duty cycle metrics for a corresponding pipeline set, generating a sliding window average of incoming jobs to the plurality of pipeline sets, and identifying a maximal position in the ordered list in which the sum of a pipeline set duty cycle metric at the maximal position minus a duty cycle metric of a subset of the plurality of duty cycle metrics up to the maximal position is greater than or equal to the sliding window average, the pipeline set duty cycle metric being in the plurality of pipeline set duty cycle metrics, for each pipeline set of the plurality of pipeline sets that is at a position less than the maximal position in the ordered list, assigning the pipeline set an assigned value calculated as a function of a difference between a duty cycle metric of the pipeline set and a duty cycle metric at the maximal position, a rate adjustment, and the sliding window average, and normalize the assigned value across the plurality of pipeline sets to obtain the plurality of weights.
This invention relates to optimizing job scheduling in a computing system with multiple processing pipelines. The problem addressed is efficiently distributing workloads across pipelines to balance utilization and prevent bottlenecks. The method involves analyzing pipeline performance metrics to assign weights that prioritize underutilized pipelines. The process begins by sorting pipeline sets based on their duty cycle metrics, which measure utilization levels. A sliding window average of incoming jobs is calculated to assess workload trends. The system identifies a maximal position in the sorted list where the cumulative duty cycle up to that point meets or exceeds the sliding window average. Pipelines before this position are assigned values based on their utilization difference from the maximal position, adjusted by a rate factor and the sliding window average. These values are normalized to generate weights, which determine job distribution priorities. This ensures workloads are dynamically routed to balance pipeline utilization and improve overall system efficiency.
11. A computer implemented system for assigning executable jobs to pipeline sets comprising: at least one aggregation thread configured to: generate a weight for each pipeline set of a plurality of pipeline sets to obtain a plurality of weights, wherein generating the weight for each pipeline set comprises: obtaining a plurality of duty cycle metrics for a plurality of pipeline threads in the pipeline set, wherein the plurality of duty cycle metrics comprises a measure of an amount of time that a corresponding pipeline thread is executing and actively processing data, and determining the weight for the pipeline set based at least in part on the plurality of duty cycle metrics; and an assigner executing on a computer processor and configured to: assign a job request to a target pipeline set selected from the plurality of pipeline sets according to a weighted random algorithm, wherein the weighted random algorithm uses the plurality of weights.
The system is designed for optimizing job assignment in a computing environment where multiple pipeline sets process executable jobs. The problem addressed is inefficient job distribution, which can lead to uneven workloads, bottlenecks, or underutilized resources. The system dynamically assigns jobs to pipeline sets based on their current processing capacity, ensuring balanced and efficient resource utilization. The system includes at least one aggregation thread that monitors the performance of pipeline sets. For each pipeline set, the aggregation thread calculates a weight by analyzing duty cycle metrics of its constituent pipeline threads. These metrics measure the time each thread spends actively processing data, providing a real-time indicator of workload. The weight for a pipeline set is derived from these metrics, reflecting its current processing capacity. An assigner component then uses these weights in a weighted random algorithm to distribute job requests. The algorithm selects a target pipeline set based on the calculated weights, favoring sets with lower current workloads. This approach ensures jobs are assigned to the most available pipeline sets, improving overall system efficiency and reducing processing delays. The system dynamically adapts to changing workloads, maintaining optimal performance across the pipeline infrastructure.
12. The computer implemented system of claim 11 , wherein generating the weight further comprises: determining an average number of data ingestion tasks assigned to the plurality of pipeline sets, wherein determining the weight is based on the average number of data ingestion tasks.
The system relates to data processing pipelines in distributed computing environments, specifically addressing the challenge of efficiently distributing data ingestion tasks across multiple pipeline sets to optimize resource utilization and performance. The invention involves a computer-implemented system that dynamically assigns weights to pipeline sets based on their workload to balance task distribution. The system monitors the number of data ingestion tasks assigned to each pipeline set and calculates an average task count across all sets. The weight for each pipeline set is then determined based on this average, ensuring that tasks are distributed proportionally to the available capacity of each set. This approach prevents overloading certain pipeline sets while underutilizing others, improving overall system efficiency and throughput. The system may also incorporate additional factors, such as pipeline set performance metrics or resource availability, to refine the weight calculation. By dynamically adjusting task distribution, the system enhances scalability and reliability in large-scale data processing environments.
13. The computer implemented system of claim 11 , wherein obtaining a duty cycle metric comprises: assigning a duty cycle ownership object to a thread, when a set of conditions is satisfied, setting the duty cycle ownership object to an on state, and processing a data object in a consumer queue while the duty cycle ownership object is in the on state, wherein the duty cycle ownership object is switched to an off state after processing the data object, wherein the set of conditions comprising the data object existing in the consumer queue of the thread, and keeping the duty cycle ownership object in the off state when the consumer queue is empty.
This invention relates to a computer-implemented system for managing thread-based data processing, particularly in environments where multiple threads compete for access to shared resources. The problem addressed is inefficient resource utilization and contention when threads process data objects from shared queues, leading to bottlenecks and reduced throughput. The system assigns a duty cycle ownership object to a thread to control its access to a consumer queue. When specific conditions are met—such as the presence of a data object in the thread's consumer queue—the duty cycle ownership object is set to an on state, allowing the thread to process the data object. Once processing is complete, the object transitions to an off state. If the consumer queue is empty, the duty cycle ownership object remains in the off state, preventing unnecessary processing attempts. This mechanism ensures fair and efficient resource allocation by dynamically granting access based on queue status, reducing contention and improving system performance. The approach is particularly useful in multi-threaded applications where workload distribution must be optimized to avoid starvation and maximize throughput.
14. The computer implemented system of claim 11 , wherein obtaining a duty cycle metric comprises: assigning a duty cycle ownership object to a thread, and when a set of conditions is satisfied, setting the duty cycle ownership object to an on state within a scope defined by determining that the set of conditions are satisfied, processing a data object in a consumer queue while within the scope, and exiting the scope to switch the duty cycle ownership object to an off state, wherein the set of conditions comprising the data object existing in the consumer queue of the thread.
This invention relates to computer-implemented systems for managing duty cycle ownership in multi-threaded environments, particularly for optimizing resource utilization and preventing contention in concurrent processing. The problem addressed is inefficient or conflicting access to shared resources when multiple threads attempt to process data objects simultaneously, leading to performance bottlenecks or race conditions. The system assigns a duty cycle ownership object to a thread, which controls access to a consumer queue. When specific conditions are met—such as the presence of a data object in the thread's consumer queue—the ownership object transitions to an "on" state, defining a processing scope. Within this scope, the thread processes the data object from the queue. Once processing is complete or the scope is exited, the ownership object switches to an "off" state, releasing control. This mechanism ensures that only one thread processes a given data object at a time, reducing contention and improving efficiency. The system dynamically manages duty cycles by monitoring queue states and adjusting ownership accordingly, preventing resource conflicts while maintaining thread independence. This approach is particularly useful in high-throughput systems where multiple threads compete for shared resources.
15. The computer implemented system of claim 11 , wherein obtaining a duty cycle metric comprises: reading a plurality of timestamps and state information from a duty cycle ownership object, and generating, as a duty cycle metric, a weighted moving average using the plurality of timestamps and the state information.
A computer-implemented system monitors and analyzes duty cycle metrics in a computing environment to optimize resource allocation and performance. The system addresses inefficiencies in tracking and managing duty cycles, particularly in multi-core or distributed systems where resource usage must be balanced dynamically. The system includes a duty cycle ownership object that records timestamps and state information, such as active, idle, or shared states, for various system components. By reading this data, the system generates a duty cycle metric as a weighted moving average, which provides a more accurate and responsive representation of resource utilization over time. This metric helps in dynamically adjusting workload distribution, reducing bottlenecks, and improving overall system efficiency. The weighted moving average accounts for recent changes more heavily, ensuring the metric reflects current conditions rather than historical averages. The system may also include mechanisms to log duty cycle data, compare metrics across components, and trigger adjustments based on predefined thresholds. This approach enhances real-time decision-making for resource management in complex computing environments.
16. The computer implemented system of claim 11 , wherein generating the weight comprises: set, for at least a subset of the plurality of pipeline sets, a pipeline set duty cycle metric as a maximal duty cycle metric of the plurality of pipeline threads in the pipeline set, generating a sliding window average of incoming jobs to the plurality of pipeline sets, and using the sliding window average and the thread set duty cycle metric to determine the weight for the pipeline set.
This invention relates to a computer-implemented system for managing workload distribution in a multi-threaded pipeline processing environment. The system addresses the challenge of efficiently allocating computational resources across multiple pipeline sets, each containing multiple pipeline threads, to optimize performance and resource utilization. The system generates a weight for each pipeline set to prioritize workload distribution. For at least a subset of the pipeline sets, the weight is determined by setting a pipeline set duty cycle metric as the highest (maximal) duty cycle metric among the pipeline threads within that set. The duty cycle metric reflects the utilization level of each thread, helping identify bottlenecks or underutilized resources. Additionally, the system calculates a sliding window average of incoming jobs directed to the pipeline sets, providing a dynamic measure of workload demand. The weight for each pipeline set is then derived from this sliding window average and the thread set duty cycle metric, ensuring that workload allocation adapts to real-time conditions. This approach balances resource allocation, preventing overloading or underutilization of pipeline threads while maintaining system efficiency. The system dynamically adjusts weights to optimize throughput and minimize latency in pipeline processing.
17. The computer implemented system of claim 11 , wherein generating the weight comprises: determine, for the pipeline set, a pipeline set duty cycle metric from the plurality of duty cycle metrics, generating a sliding window average of incoming jobs to the plurality of pipeline sets, and when the sliding window average is equal to zero, calculating a sum of the pipeline set duty cycle metric across the plurality of pipeline sets, and determining an assigned value for the pipeline set as a difference between the sum and the pipeline set duty cycle metric, normalizing the assigned value across the plurality of pipeline sets to obtain the plurality of weights.
This invention relates to a computer-implemented system for optimizing job distribution in a multi-pipeline processing environment. The system addresses the challenge of efficiently allocating jobs to multiple processing pipelines to balance workload and maximize throughput, particularly when some pipelines may be idle or underutilized. The system generates weights for each pipeline set to determine job distribution. First, it calculates a duty cycle metric for each pipeline set, representing its utilization over time. Next, it computes a sliding window average of incoming jobs across all pipeline sets. If the sliding window average is zero, indicating no incoming jobs, the system calculates the sum of all pipeline set duty cycle metrics. For each pipeline set, it then determines an assigned value by subtracting its own duty cycle metric from this sum. These assigned values are normalized across all pipeline sets to produce a set of weights. These weights are used to distribute jobs to the pipelines, ensuring fair and efficient utilization. The system dynamically adjusts job distribution based on real-time pipeline performance, preventing bottlenecks and improving overall system efficiency. The sliding window average and duty cycle metrics enable adaptive balancing, particularly in scenarios with fluctuating workloads.
18. The computer implemented system of claim 11 , wherein determining the weight for the pipeline set comprises: determine, for the pipeline set, a pipeline set duty cycle metric from the plurality of duty cycle metrics, generating a sliding window average of incoming jobs to the plurality of pipeline sets, and when the sliding window average is greater than zero, assigning a value to the pipeline set using a periodic update model to obtain an assigned value, and normalizing the assigned value across the plurality of pipeline sets to obtain the plurality of weights.
This invention relates to a computer-implemented system for optimizing job scheduling in a multi-pipeline processing environment. The system addresses the challenge of efficiently distributing workloads across multiple processing pipelines to maximize throughput and resource utilization. The invention focuses on dynamically assigning weights to pipeline sets based on their performance metrics, ensuring balanced and adaptive job allocation. The system determines a weight for each pipeline set by first calculating a pipeline set duty cycle metric from a plurality of duty cycle metrics. A sliding window average of incoming jobs is generated for the pipeline sets. If the sliding window average exceeds zero, a periodic update model assigns a value to the pipeline set, which is then normalized across all pipeline sets to obtain the weights. These weights are used to distribute jobs proportionally, improving load balancing and reducing bottlenecks. The periodic update model ensures that weights are adjusted periodically, allowing the system to adapt to changing workload conditions. The normalization step ensures that the weights are relative to each other, enabling fair distribution across all pipeline sets. This approach enhances system efficiency by dynamically optimizing resource allocation based on real-time performance data.
19. The computer implemented system of claim 11 , wherein determining the weight for the pipeline set comprises: sorting the plurality of pipeline sets in increasing order of a plurality of pipeline set duty cycle metrics to create an ordered list, wherein each pipeline set duty cycle metric is determined from the plurality of duty cycle metrics for a corresponding pipeline set, generating a sliding window average of incoming jobs to the plurality of pipeline sets, and identifying a maximal position in the ordered list in which the sum of a pipeline set duty cycle metric at the maximal position minus a duty cycle metric of a subset of the plurality of duty cycle metrics up to the maximal position is greater than or equal to the sliding window average, the pipeline set duty cycle metric being in the plurality of pipeline set duty cycle metrics, and setting an assigned value to zero for each pipeline set in the ordered list that is at a position greater than the maximal position.
The system relates to optimizing job distribution in a computing pipeline environment, addressing inefficiencies in workload balancing across multiple pipeline sets. The invention dynamically assigns weights to pipeline sets to improve resource utilization and reduce bottlenecks. The process involves analyzing duty cycle metrics for each pipeline set, which measure the proportion of time a pipeline is actively processing jobs. These metrics are aggregated to form a pipeline set duty cycle metric for each set. The pipeline sets are then sorted in increasing order of these metrics to create an ordered list. A sliding window average of incoming jobs is calculated to estimate recent workload trends. The system identifies a maximal position in the ordered list where the sum of the pipeline set duty cycle metric at that position minus the cumulative duty cycle metrics of preceding sets equals or exceeds the sliding window average. Pipeline sets beyond this maximal position are assigned a weight of zero, effectively excluding them from receiving new jobs. This ensures that only the most underutilized pipeline sets receive additional workload, balancing the system's overall efficiency. The method dynamically adjusts to changing job patterns, preventing overloading of any single pipeline set.
20. The computer implemented system of claim 11 , wherein determining the weight for the pipeline set comprises: sorting the plurality of pipeline sets in increasing order of a plurality of pipeline set duty cycle metrics to create an ordered list, wherein each pipeline set duty cycle metric is determined from the plurality of duty cycle metrics for a corresponding pipeline set, generating a sliding window average of incoming jobs to the plurality of pipeline sets, and identifying a maximal position in the ordered list in which the sum of a pipeline set duty cycle metric at the maximal position minus a duty cycle metric of a subset of the plurality of duty cycle metrics up to the maximal position is greater than or equal to the sliding window average, the pipeline set duty cycle metric being in the plurality of pipeline set duty cycle metrics, for each pipeline set of the plurality of pipeline sets that is at a position less than the maximal position in the ordered list, assigning the pipeline set an assigned value calculated as a function of a difference between a duty cycle metric of the pipeline set and a duty cycle metric at the maximal position, a rate adjustment, and the sliding window average, and normalize the assigned value across the plurality of pipeline sets to obtain the plurality of weights.
This invention relates to a computer-implemented system for optimizing job distribution in a multi-pipeline processing environment. The system addresses the problem of efficiently allocating incoming jobs to multiple processing pipelines to balance workload and maximize throughput. The system determines weights for each pipeline set based on their duty cycle metrics, which represent the utilization or workload of each pipeline. The process involves sorting pipeline sets by their duty cycle metrics in ascending order to create an ordered list. A sliding window average of incoming jobs is then calculated to assess recent workload trends. The system identifies a maximal position in the ordered list where the sum of the duty cycle metric at that position minus the sum of metrics up to that position meets or exceeds the sliding window average. For pipeline sets positioned before the maximal position, the system assigns a value based on the difference between their duty cycle metric and the metric at the maximal position, adjusted by a rate factor and the sliding window average. These values are then normalized across all pipeline sets to generate the final weights. These weights are used to distribute incoming jobs proportionally, ensuring balanced utilization and preventing bottlenecks. The system dynamically adapts to changing workloads by continuously updating the weights based on real-time duty cycle metrics.
21. A non-transitory computer readable medium comprising computer readable program code for performing operations, the operations comprising: generating a weight for each pipeline set of a plurality of pipeline sets to obtain a plurality of weights, wherein generating the weight for each pipeline set comprises: obtaining a plurality of duty cycle metrics for a plurality of pipeline threads in the pipeline set, wherein the plurality of duty cycle metrics comprises a measure of an amount of time that a corresponding pipeline thread is executing and actively processing data, and determining the weight for the pipeline set based on the plurality of duty cycle metrics; and assigning a new job request to a target pipeline set selected from the plurality of pipeline sets according to a weighted random algorithm, wherein the weighted random algorithm uses the plurality of weights.
This invention relates to optimizing job assignment in a multi-pipeline processing system. The problem addressed is inefficient resource utilization in systems where multiple processing pipelines handle concurrent tasks, leading to uneven workload distribution and potential bottlenecks. The solution involves dynamically assigning jobs to pipelines based on their current activity levels to improve load balancing and throughput. The system generates a weight for each pipeline set in a plurality of pipeline sets. For each pipeline set, a plurality of duty cycle metrics are obtained for the pipeline threads within that set. These metrics measure the amount of time each thread spends actively executing and processing data. The weight for the pipeline set is then determined based on these duty cycle metrics, reflecting the set's current workload. A new job request is assigned to a target pipeline set selected from the plurality of pipeline sets using a weighted random algorithm that incorporates the generated weights. This ensures that jobs are distributed more evenly across pipelines, reducing idle time and improving overall system efficiency. The approach dynamically adapts to changing workload conditions, enhancing performance in real-time processing environments.
22. The non-transitory computer readable medium of claim 21 , wherein generating the weight further comprises: determining an average number of data ingestion tasks assigned to the plurality of pipeline sets, wherein determining the weight is based on the average number of data ingestion tasks.
The invention relates to a system for optimizing data processing in a distributed computing environment, specifically addressing the challenge of efficiently distributing data ingestion tasks across multiple pipeline sets to balance workload and improve performance. The system involves dynamically assigning weights to pipeline sets based on their current workload, where the weight is determined by calculating the average number of data ingestion tasks assigned to each pipeline set. This weight is then used to adjust the distribution of new tasks, ensuring that pipeline sets with fewer tasks receive more assignments, thereby preventing bottlenecks and improving overall system efficiency. The method includes monitoring task assignments, calculating the average task load per pipeline set, and using this metric to generate a weight that influences task distribution. The system may also incorporate additional factors, such as pipeline set capacity or historical performance, to further refine the weighting process. By dynamically balancing the workload across pipeline sets, the invention enhances resource utilization and reduces processing delays in large-scale data ingestion environments.
23. The non-transitory computer readable medium of claim 21 , wherein obtaining a duty cycle metric comprises: assigning a duty cycle ownership object to a thread, when a set of conditions is satisfied, setting the duty cycle ownership object to an on state, and processing a data object in a consumer queue while the duty cycle ownership object is in the on state, wherein the duty cycle ownership object is switched to an off state after processing the data object, wherein the set of conditions comprising the data object existing in the consumer queue of the thread, and keeping the duty cycle ownership object in the off state when the consumer queue is empty.
This invention relates to a computer-implemented method for managing thread execution in a multi-threaded system, specifically addressing the problem of efficiently distributing processing workloads among threads while minimizing idle time and resource contention. The system assigns a duty cycle ownership object to a thread when specific conditions are met, such as the presence of a data object in the thread's consumer queue. Once assigned, the duty cycle ownership object is set to an on state, allowing the thread to process the data object. After processing, the object transitions to an off state. If the consumer queue is empty, the duty cycle ownership object remains in the off state, preventing unnecessary thread activation. This mechanism ensures that threads only process data when available, reducing overhead and improving system efficiency. The duty cycle ownership object acts as a control mechanism to regulate thread activity dynamically, ensuring optimal resource utilization and minimizing contention in shared environments. The invention is particularly useful in high-throughput systems where efficient workload distribution is critical.
24. The non-transitory computer readable medium of claim 21 , wherein obtaining a duty cycle metric comprises: assigning a duty cycle ownership object to a thread, and when a set of conditions is satisfied, setting the duty cycle ownership object to an on state within a scope defined by determining that the set of conditions are satisfied, processing a data object in a consumer queue while within the scope, and exiting the scope to switch the duty cycle ownership object to an off state, wherein the set of conditions comprising the data object existing in the consumer queue of the thread.
This invention relates to computer systems managing thread-based processing of data objects, particularly in scenarios where multiple threads compete for access to shared resources. The problem addressed is ensuring efficient and fair distribution of processing workloads among threads while preventing resource contention and deadlocks. The invention involves a method for managing thread execution using a duty cycle metric. A duty cycle ownership object is assigned to a thread, which controls access to a consumer queue containing data objects. When specific conditions are met—such as the presence of a data object in the thread's consumer queue—the duty cycle ownership object transitions to an "on" state, enabling the thread to process the data object within a defined scope. Once processing is complete, the thread exits the scope, switching the duty cycle ownership object to an "off" state. This mechanism ensures that threads only process data objects when they are actively assigned ownership, reducing contention and improving resource utilization. The system dynamically adjusts ownership based on queue status, ensuring fair and efficient workload distribution.
25. The non-transitory computer readable medium of claim 21 , wherein obtaining a duty cycle metric comprises: reading a plurality of timestamps and state information from a duty cycle ownership object, and generating, as a duty cycle metric, a weighted moving average using the plurality of timestamps and the state information.
This invention relates to monitoring and analyzing duty cycle metrics in computing systems, particularly for tracking resource usage and ownership over time. The problem addressed involves accurately measuring and quantifying duty cycle behavior, which is essential for performance optimization, load balancing, and resource allocation in distributed or multi-threaded environments. The invention provides a method for obtaining a duty cycle metric by reading a plurality of timestamps and state information from a duty cycle ownership object. The timestamps and state information represent the periods during which a resource or process was active or idle. The system then generates a weighted moving average using this data to produce a duty cycle metric. The weighted moving average allows for dynamic adjustments in the metric calculation, giving more importance to recent data points while still considering historical trends. This approach improves accuracy in tracking duty cycle behavior, especially in systems where resource usage patterns change over time. The duty cycle ownership object stores the timestamps and state information, which may include details such as the start and end times of active periods, the identity of the owner, and the resource state during those intervals. By analyzing this data, the system can determine how frequently and for how long a resource was in use, enabling better decision-making for resource management. The weighted moving average ensures that the metric reflects both short-term fluctuations and long-term trends, providing a balanced view of duty cycle performance. This method is particularly useful in high-performance computing, cloud environments, and real-time systems where efficient resource utilization is critical.
26. The non-transitory computer readable medium of claim 21 , wherein generating the weight comprises: set, for at least a subset of the plurality of pipeline sets, a pipeline set duty cycle metric as a maximal duty cycle metric of the plurality of pipeline threads in the pipeline set, generating a sliding window average of incoming jobs to the plurality of pipeline sets, and using the sliding window average and the thread set duty cycle metric to determine the weight for the pipeline set.
This invention relates to optimizing job scheduling in a multi-threaded pipeline processing system. The problem addressed is efficiently distributing workloads across multiple pipeline sets to balance resource utilization and minimize latency. The system includes multiple pipeline sets, each containing multiple pipeline threads, where each thread processes jobs in a sequential manner. The invention improves upon prior art by dynamically adjusting weights assigned to each pipeline set based on real-time performance metrics. The method involves calculating a duty cycle metric for each pipeline set, defined as the maximum duty cycle among all threads within the set. This metric reflects the most heavily utilized thread in the set, ensuring that no single thread becomes a bottleneck. Additionally, a sliding window average of incoming jobs is computed to assess workload trends over time. The weight for each pipeline set is then determined by combining the duty cycle metric and the sliding window average, allowing the scheduler to prioritize pipeline sets that are either underutilized or experiencing sudden workload spikes. This adaptive weighting mechanism ensures balanced resource allocation and prevents overloading any single pipeline set. The invention is implemented as a non-transitory computer-readable medium containing instructions for executing the described scheduling logic.
27. The non-transitory computer readable medium of claim 21 , wherein generating the weight comprises: determine, for the pipeline set, a pipeline set duty cycle metric from the plurality of duty cycle metrics, generating a sliding window average of incoming jobs to the plurality of pipeline sets, and when the sliding window average is equal to zero, calculating a sum of the pipeline set duty cycle metric across the plurality of pipeline sets, and determining an assigned value for the pipeline set as a difference between the sum and the pipeline set duty cycle metric, normalizing the assigned value across the plurality of pipeline sets to obtain the plurality of weights.
This invention relates to optimizing job scheduling in a computing system with multiple pipeline sets. The problem addressed is efficiently distributing workloads across pipelines to balance resource utilization and prevent bottlenecks. The solution involves dynamically generating weights for pipeline sets based on their duty cycle metrics and job inflow patterns. The system first determines a duty cycle metric for each pipeline set, representing its utilization over time. It then calculates a sliding window average of incoming jobs across all pipeline sets. If the sliding window average is zero, indicating no new jobs are arriving, the system computes a sum of all pipeline set duty cycle metrics. For each pipeline set, it assigns a value equal to the difference between this total sum and the pipeline set's own duty cycle metric. This value is then normalized across all pipeline sets to generate a set of weights. These weights are used to distribute jobs more evenly, ensuring no single pipeline becomes overloaded while others remain underutilized. The approach adapts to varying workloads by continuously recalculating weights based on real-time metrics, improving overall system efficiency and throughput.
28. The non-transitory computer readable medium of claim 21 , wherein determining the weight for the pipeline set comprises: determine, for the pipeline set, a pipeline set duty cycle metric from the plurality of duty cycle metrics, generating a sliding window average of incoming jobs to the plurality of pipeline sets, and when the sliding window average is greater than zero, assigning a value to the pipeline set using periodic update model to obtain an assigned value, and normalizing the assigned value across the plurality of pipeline sets to obtain the plurality of weights.
This invention relates to optimizing job scheduling in a computing system with multiple pipeline sets. The problem addressed is efficiently distributing workloads across pipelines to maximize throughput and resource utilization while minimizing bottlenecks. The solution involves dynamically assigning weights to pipeline sets based on their performance metrics and current workload. The system determines a pipeline set duty cycle metric, which measures the proportion of time the pipeline is actively processing jobs. A sliding window average of incoming jobs is generated to assess recent workload trends. If the sliding window average exceeds zero, indicating active job processing, a periodic update model assigns a value to the pipeline set. This value is then normalized across all pipeline sets to produce a set of weights. These weights are used to distribute incoming jobs proportionally, ensuring that pipelines with higher capacity or lower current load receive more jobs, thereby balancing the workload and improving overall system efficiency. The periodic update model allows for real-time adjustments, adapting to changing workload patterns. Normalization ensures fair distribution, preventing any single pipeline from being overloaded while others remain underutilized. This approach enhances system performance by dynamically optimizing resource allocation based on real-time metrics.
29. The non-transitory computer readable medium of claim 21 , wherein determining the weight for the pipeline set comprises: sorting the plurality of pipeline sets in increasing order of a plurality of pipeline set duty cycle metrics to create an ordered list, wherein each pipeline set duty cycle metric is determined from the plurality of duty cycle metrics for a corresponding pipeline set, generating a sliding window average of incoming jobs to the plurality of pipeline sets, and identifying a maximal position in the ordered list in which the sum of a pipeline set duty cycle metric at the maximal position minus a duty cycle metric of a subset of the plurality of duty cycle metrics up to the maximal position is greater than or equal to the sliding window average, the pipeline set duty cycle metric being in the plurality of pipeline set duty cycle metrics, and setting an assigned value to zero for each pipeline set in the ordered list that is at a position greater than the maximal position.
The invention relates to optimizing job scheduling in a computing system with multiple processing pipelines. The problem addressed is efficiently distributing workloads across pipeline sets to balance resource utilization and prevent bottlenecks. The solution involves dynamically assigning weights to pipeline sets based on their duty cycle metrics and incoming job rates. The method sorts pipeline sets by their duty cycle metrics in ascending order, creating an ordered list. Each pipeline set's duty cycle metric is derived from individual duty cycle metrics of its constituent pipelines. A sliding window average of incoming jobs is calculated to estimate current workload trends. The system then identifies a maximal position in the ordered list where the sum of the pipeline set's duty cycle metric minus the cumulative duty cycle metrics of preceding sets equals or exceeds the sliding window average. Pipeline sets beyond this maximal position are assigned a weight of zero, effectively excluding them from receiving new jobs. This ensures that heavily utilized pipelines are not overloaded while underutilized ones receive more tasks, improving overall system efficiency and throughput. The approach dynamically adapts to changing workload patterns, maintaining balanced resource allocation.
30. The non-transitory computer readable medium of claim 21 , wherein determining the weight for the pipeline set comprises: sorting the plurality of pipeline sets in increasing order of a plurality of pipeline set duty cycle metrics to create an ordered list, wherein each pipeline set duty cycle metric is determined from the plurality of duty cycle metrics for a corresponding pipeline set, generating a sliding window average of incoming jobs to the plurality of pipeline sets, and identifying a maximal position in the ordered list in which the sum of a pipeline set duty cycle metric at the maximal position minus a duty cycle metric of a subset of the plurality of duty cycle metrics up to the maximal position is greater than or equal to the sliding window average, the pipeline set duty cycle metric being in the plurality of pipeline set duty cycle metrics, for each pipeline set of the plurality of pipeline sets that is at a position less than the maximal position in the ordered list, assigning the pipeline set an assigned value calculated as a function of a difference between a duty cycle metric of the pipeline set and a duty cycle metric at the maximal position, a rate adjustment, and the sliding window average, and normalize the assigned value across the plurality of pipeline sets to obtain the plurality of weights.
This invention relates to optimizing job scheduling in a computing system with multiple processing pipelines. The problem addressed is efficiently distributing workloads across pipelines to balance utilization and prevent bottlenecks. The solution involves dynamically assigning weights to pipeline sets based on their duty cycle metrics, which measure how busy each pipeline is. The process begins by sorting pipeline sets in increasing order of their duty cycle metrics, creating an ordered list. A sliding window average of incoming jobs is then calculated to estimate recent workload trends. The system identifies a maximal position in the ordered list where the sum of a pipeline set's duty cycle metric minus the cumulative duty cycle metrics up to that position meets or exceeds the sliding window average. For pipeline sets before this maximal position, weights are assigned based on the difference between their duty cycle and the duty cycle at the maximal position, adjusted by a rate factor and the sliding window average. These weights are then normalized across all pipeline sets to ensure balanced distribution. This approach dynamically adjusts workload allocation to maintain optimal pipeline utilization.
Unknown
December 1, 2020
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.