Methods and apparatus for workflow processing

This application claims the benefit of priority under 35 U.S.C 119 (e) to U.S. Provisional Patent Application No. 63/423,797 filed on Nov. 8, 2022 and entitled “Vector Work Queue Entry (VWQE) Search using Cuckoo Hashing Algorithm,” which is incorporated by reference herein in its entirety.

The exemplary embodiments of the present invention relate to telecommunications networks. More specifically, the exemplary embodiments of the present invention relate to receiving and processing workflows via a wireless communication network.

2 3 One of the key components of virtually any layeror layerswitch is the Ternary Content Addressable Memory (TCAM). TCAM is an expensive component and is typically a scarce resource on many switching platforms. This especially holds true on lower-end platforms where the administrator must make decisions about how to allocate the available TCAM resources to match the role the switch plays in the network.

Unfortunately, TCAM is not the cure all for hardware search lookups since TCAM is expensive, power hungry and takes up quite a bit of silicon space.

In various exemplary embodiments, methods and apparatus are provided for workflow processing. In an embodiment, work queue entries (WQE) are aggregate utilizing an SRAM memory and hashing algorithm. Each WQEs comprises data and a key value. The key values are hashed to obtain addresses with which to access the SRAM memory. Aggregation queues are also provided to allow WQE's having the same key value to be aggregated to form vectors of WQE's (VWQE).

In an embodiment, a type of hashing referred to “Cuckoo” hashing is utilized. Cuckoo hashing is a form of open addressing in which each non-empty cell of a hash table contains a key or key-value pair. A hash function is used to determine the location for each key, and its presence in the table (or the value associated with it) can be found by examining that cell of the table. However, open addressing suffers from collisions, which happens when more than one key is mapped to the same cell. The basic idea of cuckoo hashing is to resolve collisions by using two hash functions instead of only one. This provides two possible locations in the hash table for each key.

In an embodiment, a method is provided that includes receiving a workflow queue entry (WQE) that includes a key value and searching a plurality of memory tables for an address that contains the key value. When the address that contains the key value is found, obtaining a first queue identifier from the address. When the address that contains the key value is not found, storing the key value and a second queue identifier at a selected address in a selected memory table. The method also includes storing the WQE in an aggregation queue identified by one of the first and second queue identifiers, updating a count associated with the aggregation queue, and when the count reached a selected level, aggregating all WQE entries in the aggregation queue to form a vector WQE.

In another embodiment, an apparatus is provided that includes an interface configured to receive a workflow queue entry (WQE) that includes a key value, a plurality of memory tables, and an aggregation table having one or more aggregation queues. The apparatus also includes a processor configured to perform operations of searching the plurality of memory tables for an address that contains the key value, when the address that contains the key value is found, obtaining a first queue identifier from the address, and when the address that contains the key value is not found, storing the key value and a second queue identifier at a selected address in a selected memory table. The processor is also configured to perform operations of storing the WQE in an aggregation queue identified by one of the first and second queue identifiers, updating a count associated with the aggregation queue, and when the count reached a selected level, aggregating all WQE entries in the aggregation queue to form a WQE vector (VWQE).

Additional features and benefits of the exemplary embodiments of the present invention will become apparent from the detailed description, figures and claims set forth below.

The purpose of the following detailed description is to provide an understanding of one or more embodiments of the present invention. Those of ordinary skills in the art will realize that the following detailed description is illustrative only and is not intended to be in any way limiting. Other embodiments will readily suggest themselves to such skilled persons having the benefit of this disclosure and/or description.

In the interest of clarity, not all of the routine features of the implementations described herein are shown and described. It will, of course, be understood that in the development of any such actual implementation, numerous implementation-specific decisions may be made in order to achieve the developer's specific goals, such as compliance with application and/or business-related constraints, and that these specific goals will vary from one implementation to another and from one developer to another. Moreover, it will be understood that such a development effort might be complex and time-consuming but would nevertheless be a routine undertaking of engineering for those of ordinary skills in the art having the benefit of embodiments of this disclosure.

Various embodiments of the present invention illustrated in the drawings may not be drawn to scale. Rather, the dimensions of the various features may be expanded or reduced for clarity. In addition, some of the drawings may be simplified for clarity. Thus, the drawings may not depict all of the components of a given apparatus (e.g., device) or method. The same reference indicators will be used throughout the drawings and the following detailed description to refer to the same or like parts.

The term “system” or “device” is used generically herein to describe any number of components, elements, sub-systems, devices, packet switch elements, packet switches, access switches, routers, networks, modems, base stations, eNB (eNodeB), computer and/or communication devices or mechanisms, or combinations of components thereof. The term “computer” includes a processor, memory, and buses capable of executing instruction wherein the computer refers to one or a cluster of computers, personal computers, workstations, mainframes, or combinations of computers thereof.

1 FIG. 1 FIG. 100 106 108 102 1 106 104 102 1 1 106 116 124 118 106 shows a communication networkcomprising an enterprise serverthat includes an embodiment of a workflow processor. As illustrated in, a plurality of clients(-N) communicate with the enterprise serverthrough communication network. The clients(-N) transmit workflows (-N) to the enterprise server. In an embodiment, the workflows comprise work queue entries (WQE)that include dataand a key. During operation, thousands, or millions of WQE's are received at the enterprise server.

108 110 112 114 126 110 118 216 120 120 The workflow processorcomprises controller, hash circuit, SRAM memory, and aggregator. The controllerreceives the keys of the WQE's, such as key, and processes the received keys to control the aggregatorto generate workflow vectors VWQEthat comprise a string or stream of WQE's having the same key. The VWQE'sare transmitted to downstream processing systems for further processing.

112 114 126 120 The hash circuitoperates to hash the keys to generate address values that are used to access the SRAM. For example, in one embodiment, the hashing algorithm comprises a cuckoo hashing algorithm to resolve hash collisions. The keys are then stored in the SRAM at the hashed addresses. The keys are stored with an index to a queue in the aggregator. The WQE's are stored in the aggregator in the corresponding queue. Once an aggregator queue is filled, the WQE's within the queue are assembled into a VWQEfor output.

120 108 Since the workflow processor utilizes SRAM for storage, the result is a smaller and power efficient search engine that is able to process the received WQE's to form the VWQE's. Additional descriptions of the workflow processorand its operations are provided below.

2 FIG. 1 FIG. 200 200 108 200 202 204 206 208 210 212 214 shows an exemplary embodiment of a workflow processor. For example, the workflow processoris suitable for use as the workflow processorshown in. The workflow processorcomprises processor, instruction memory, SRAM memory, aggregation memory, output interface, and network interfaceall coupled to communicate using bus.

202 The processorcomprises at least one of a CPU, processor, state machine, programmable logic, memory, hardware, and/or discrete components configured to perform the functions described herein.

204 204 216 202 The instruction memorycomprises any suitable memory, such as RAM, PROM, or other type of memory. In one embodiment, the memorystores a hash algorithmthat can be retrieved by the processorand executed to perform a hash function as described herein.

206 The SRAMcomprises one or more static RAM devices that are used to store received keys along with aggregation queue identifiers, timestamps, and other relevant information.

208 116 120 208 218 The aggregation memorycomprises any suitable memory, such as RAM, or other type of memory that forms queues to store WQE'sthat are combined to form the VWQE. In one embodiment, the aggregation memorystores an aggregation tableof queues of WQE's associated with different keys.

212 212 116 104 116 102 1 1 FIG. The network interfacecomprises at least one of a CPU, processor, state machine, programmable logic, memory, hardware, and/or discrete components that perform the functions described herein. The network interfaceoperates to receive WQEsthat are received from a communication network, such as the network. The WQE'sare transmitted by one or more client devices, such as clients(-N) shown in.

210 120 202 210 218 120 The output interfacecomprises at least one of registers, gates, latches, hardware components, and/or discrete components that output the VWQEworkflow results. For example, in an exemplary embodiment, the processorcontrols the output interfaceto output WQE's stored in the aggregation tableas the VWQEworkflow results.

116 212 202 218 210 218 202 210 120 200 During operation, WQE'sreceived by the network interfaceare processed by the processor. In an embodiment, for each received WQE, the key portion is used to search the SRAM. If the key portion matches a stored key, then the WQE is stored in an aggregation queue identified by the stored key. The aggregation queue is located in the aggregation table. If the key portion does not match a key stored in the SRAM, then the key portion is hashed to obtain an address. The address is used to access the SRAM tables to find an empty location. When an empty location is found, the key is stored in the empty location along with an aggregation queue identifier and a time stamp. If no empty locations are found, the address of the SRAM table having the oldest key is used to store the new key and the oldest key is deleted. The aggregation queue associated with the oldest key is closed or terminated and the WQE's in the closed aggregation queue are combined to form a VWQE that is output from the output interface. As additional keys are processed, the queues of the aggregation tablefill up and the processorextracts the WQE's from full aggregation queues to form workflow vectors that are output by the output interfaceas the VWQE. A more detailed description of the design and operation of the workflow processoris provided below.

3 FIG. 2 FIG. 3 FIG. 206 300 0 302 304 306 308 shows exemplary embodiments of SRAM and aggregation tables for use with the workflow processor shown in. As illustrated in, the SRAM memoryis divided into “N” SRAM tables. In an embodiment, N is any integer value, for instance, in one embodiment, the value of N is four. Table addresses (-Y) are used to access the tables at the same time, or each table can be individually accessed. For example, in an embodiment, the table address range is 0-256, which can be addressed using an 8-bit address. In an embodiment, a received key is processed by a hash algorithm to produce a hashed address that is used to access the tables. For example, the hashed address HI (x)is used to access corresponding locations in all N tables. For each table, the entry stored at the location accessed by the hashed address includes an aggregation queue number (N), a key, and a timestamp (T).

310 218 310 218 218 1 310 202 During operation, entries in the tables of the SRAM are updated as additional keys are received. A countis maintained to indicate the number of WQE entries in each aggregation queue. For example, as shown at aggregation table, the counta count is stored with each queue in the aggregation table. When the count for a particular queue reaches a certain level, the WQE entries are aggregated into a vector (VWQE) and output. The queue is then cleared and ready to accept more entries. For example, five WQE entries are stored in the aggregation tableat queue numberand the countshows a value of 5. When an aggregation queue is filled, the processoraggregates the WQE entries of that queue and forms them into a vector (VWQE) that is transmitted to a downstream processing system.

4 FIG. 400 400 108 200 shows an exemplary methodfor storing workflow keys as performed by embodiments of a workflow processor. For example, methodis performed by embodiments of the workflow processoror the workflow processor.

402 104 212 202 At block, a new WQE is received. For example, the new WQE is transmitted by a client over the communication networkand received by the network interface. The received WQE is then passed to the processorfor processing.

404 202 206 At block, tables in the SRAM are searched for the key that is part of the new WQE. In an embodiment, the processorsearches the contents of the tables in the SRAMfor the key of the new WQE.

406 202 206 422 206 408 At block, a determination is made as to whether the key was found in the tables of the SRAM. For example, the processormakes this determination by comparing the contents of each SRAM table entry with the key. If the key is found in the tables of the SRAM, the method proceeds to block. If the key is not found in the tables of the SRAM, the method proceeds to block.

408 202 218 206 At block, a hash function is performed on the key to obtain a hash value that represents an address. For example, the processorperforms the hash algorithmto hash the key to obtain the hash value. The hash value represents a hashed address that will be used to access the tables of the SRAM.

410 202 412 414 At block, a determination is made as to whether there is an open location in the tables of the SRAM at the hashed address. For example, the processorevaluates the contents of the tables in the SRAM at the hashed address to determine if there is an open or unused location. Each of the “N” SRAM tables is checked. If there is an unused location at the hashed address the method proceeds to block. If there is not an open location in any of the tables of the SRAM at the hashed address, the method proceeds to block.

412 202 At block, the key is stored at open location in the tables of the SRAM at the hashed address. In addition to the key, an aggregation queue number and timestamp are also stored with the key in the table of the SRAM. In an embodiment, the processoridentifies a queue in the aggregation table to be used for WQE's having this key.

414 202 202 310 At block, the received WQE is stored in the identified aggregation queue. For example, the processorstores the received WQE in the aggregation queue that corresponds to the queue identifier. The processoralso increments the countthat indicates the number of WQE's in the aggregation queue.

416 202 308 304 202 At block, the oldest key stored in the tables of the SRAM at the hashed address is removed and the WQE's in the corresponding queue in the aggregation table are aggregated into a VWQE and output. For example, the processordetermines the oldest key stored in the tables of the SRAM at the hashed address by reading and comparing the associated timestamps. The processor also obtains the aggregation queue numberof the oldest key from the tables of the SRAM. The processorthen deletes this oldest key and associated entries in the SRAM and aggregates the WQE entries in the corresponding aggregation queue number to form a VWQE that is output.

418 202 At block, the new key is stored in the tables of the SRAM at the hashed address and location that the oldest key was found. For example, the processorstores the new key to replace the oldest key in the tables of the SRAM.

420 202 At block, in addition to storing the new key at the location of the oldest key, the processorstores a new queue number and timestamp for the new key.

422 202 202 At block, after finding a key in the tables of the SRAM that matches the new key, the received WQE is stored in the aggregation queue associated with the found key. For example, the processorreads the found key from the tables of the SRAM in addition to its associated aggregation queue number and timestamp. The processorthen stores the new WQE in the identified queue of the aggregation table.

424 202 At block, the count of the aggregation queue is updated to reflect that a new WQE has been added to the aggregation queue. For example, the processorupdates the count associated with the aggregation queue in which the new WQE is stored.

426 202 308 At block, the timestamp of the key in the tables of the SRAM is updated. For example, the processorupdates the timestampof the key found in the SRAM tables to reflect the timestamp of the newly received WQE.

400 400 Thus, the methodoperates to store workflow keys received by a workflow processor. It should be noted that the operations of methodare exemplary and not limiting and that the operations can be added to, deleted, updated, rearranged, or otherwise modified within the scope of the embodiments.

5 FIG. 500 500 108 200 shows an exemplary methodfor storing received WQE's in an aggregation table. For example, methodis performed by embodiments of the workflow processoror the workflow processor.

502 202 At block, an aggregation queue for a received WQE is identified. In one embodiment, the tables of the SRAM are searched for the key associated with the WQE. If the key is found, the aggregation queue stored with the key is the aggregation queue to be used for the WQE. If the key is not found during the search, an empty aggregation queue is used for the WQE. In an embodiment, the processorassigns the aggregation queue number to the WQE.

504 202 506 512 At block, a determination is made as to whether there is space in the assigned queue of the aggregation table to store the WQE. For example, the assigned queue in the aggregation table may be full. The processormakes this determination. If there is space in the assigned queue for the WQE, the method proceeds to block. If there is not space in the assigned queue for the WQE, the method proceeds to block.

506 202 At block, the WQE is stored in the assigned queue in the aggregation table. For example, the processorstores the received WQE in the assigned queue of the aggregation table.

508 202 308 At block, a timestamp stored with the key in the table of the SRAM is updated with the timestamp of the new WQE. For example, the processorupdates the timestampin the table of the SRAM with the timestamp associated with the received WQE.

510 202 310 At block, a count stored with the queue of the aggregation table is updated with the entry of the WQE into the queue. For example, the processorupdates the countassociated with the queue in the aggregation table to reflect that a new WQE has been stored in the queue.

512 202 At block, a vector of WQE entries from the aggregation queue is formed. For example, the processoraggregates the WQE entries from the assigned queue in the aggregation table to form a VWQE.

514 202 210 At block, the vector VWQE is output. For example, the processoroutputs the VWQE to downstream processing system using the output interface.

516 202 At block, entries in the assigned queue of the aggregation table are deleted and the count is cleared. For example, the processorperforms this function.

500 500 Thus, the methodoperates to store workflow WQE's received by a workflow processor into an aggregation table. It should be noted that the operations of methodare exemplary and not limiting and that the operations can be added to, deleted, updated, rearranged, or otherwise modified within the scope of the embodiments.

6 FIG. 600 202 shows an exemplary methodfor forming a VWQE output from WQE's in an aggregation queue. For example, in an embodiment, the processorreads WQE's stored in a queue of the aggregation table and forms a vector (VWQE) for output.

602 202 At block, an aggregation queue in the aggregation table is identified. For example, the processoridentifies the aggregation queue as being full or having entries that are getting to old.

604 202 At block, WQE's stored in the aggregation queue are read. For example, the processorreads the WQE's from the identified aggregation queue.

606 202 At block, the WQE's that are read from the aggregation queue are aggregated to form a vector VWQE. For example, the processoraggregates the WQE's read from the identified queue and aggregates these WQE's to form a vector VWQE.

608 202 210 At block, the vector VWQE is output. For example, the processoroutputs the VWQE from the output interface.

610 202 At block, the entries of WQE's stored in the aggregation queue are deleted and the associated count is cleared. For example, the processorperforms this function.

600 600 Thus, the methodoperates to form a VWQE output from WQE's stored in an aggregation queue. It should be noted that the operations of methodare exemplary and not limiting and that the operations can be added to, deleted, updated, rearranged, or otherwise modified within the scope of the embodiments.

7 FIG. 700 700 108 200 shows an exemplary methodfor scrubbing an aggregation table. For example, methodis performed by embodiments of the workflow processoror the workflow processor.

702 At block, a first queue of an aggregation table is selected.

704 202 310 706 712 At block, determination is made as to whether the queue is filled. For example, the processormakes this determination by checking the value of the associated count. If the queue is not filled, the method proceeds to block. If the queue is filled or at a selected number of entries, the method proceeds to block.

706 202 708 712 At block, a determination is made as to whether the entries are too old. For example, the processormakes this determination by checking the timestamp of the queue's corresponding key stored in tables of the SRAM. If the entries in the queue are not too old, the method proceeds to block. If the entries in the queue are too old, the method proceeds to block.

708 202 710 At block, a determination is made as to whether all queues have been checked. The processormakes this determination. If all the queues have been checked, the method ends. If all the queues have not been checked, the method proceeds to block.

710 202 At block, the next queue in the aggregation table is selected. For example, the processorselects the next queue in the aggregation table to scrub.

712 202 202 210 At block, a vector of WQE (VWQE) is formed from the entries in the selected queue. For example, the processorforms the VWQE from the WQE entries stored in the selected queue. The processorthen outputs the VWQE using the output interface.

714 202 At block, the WQE entries in the selected queue are deleted and the count associated with the queue is cleared. For example, the processorperforms this operation.

700 700 Thus, the methodoperates to scrub an aggregation table. It should be noted that the operations of methodare exemplary and not limiting and that the operations can be added to, deleted, updated, rearranged, or otherwise modified within the scope of the embodiments.

8 FIG. 800 108 200 shows an exemplary method for scrubbing SRAM tables. For example, methodis performed by embodiments of the workflow processoror the workflow processor.

802 202 At block, a first SRAM table is selected. For example, the processorselects the first SRAM table for scrubbing.

804 202 At block, a first address of the selected SRAM table is selected. For example, the processorselects the first address of the selected SRAM table.

806 202 816 808 At block, the determination is made as to whether the entry in the SRAM address is too old. For example, if the address includes a key value, the timestamp associated with the key value is checked to determine if the entry is too old. The processorperforms this operation. If the key value is too old, the method proceeds to block. If the key value is not too old, the method proceeds to block.

808 202 812 810 At block, a determination is made as to whether all SRAM addresses in the selected SRAM table have been checked. The processormakes this determination. If all the addresses in the selected SRAM table have been checked, the method proceeds to block. If all the addresses have not been checked, the method proceeds to block.

810 202 At block, the next address in the SRAM table is selected. The processorperforms this operation.

812 202 814 At block, a determination is made as to whether all SRAM tables have been checked. The processorperforms this operation. If all SRAM tables have been checked, the method ends. If all SRAM tables have not been checked, the method proceeds to block.

814 202 At block, the next SRAM table is selected. The processorperforms this operation.

816 202 At block, the WQE entries in the queue of the aggregation table that is associated with the SRAM entry is aggregated to form a VWQE that is output. The processorperforms this operation. moved to the aggregation table

818 At block, the entry is deleted from the SRAM at the selected address and the timestamp and queue identifier are also deleted.

800 800 Thus, the methodoperates to scrub SRAM tables. It should be noted that the operations of methodare exemplary and not limiting and that the operations can be added to, deleted, updated, rearranged, or otherwise modified within the scope of the embodiments.

The exemplary aspect of the present invention includes various processing steps as described above. The steps may be embodied in machine or computer executable instructions. The instructions can be used to cause special purpose system, which is programmed with the instructions, to perform the steps of the exemplary embodiment of the present invention. Alternatively, the steps of the exemplary embodiment of the present invention may be performed by specific hardware components that contain hard-wired logic for performing the steps, or by any combination of programmed computer components and custom hardware components.

While particular embodiments of the present invention have been shown and described, it will be obvious to those skilled in the art that, based upon the teachings herein, changes and modifications may be made without departing from these exemplary embodiments and their broader aspects. Therefore, the appended claims are intended to encompass within their scope all such changes and modifications as are within the true spirit and scope of these exemplary embodiments of the present invention.