Flow Based Dynamic Load Balancing for Cost Effective Switching Systems

1. An apparatus for load balancing packets within a network device, said network device having a plurality of network device sub-systems, each said sub-system having at least one network device system port and a plurality of fabric ports, said fabric ports connecting said each sub-system with others of said sub-systems within said network device, said apparatus comprising: a classifier, said classifier configured to associate each of said packets with a flow identifier, said flow identifier indicating a network flow to which each of said packets belongs; a plurality of queues, each of said fabric ports having one of said queues associated therewith, each said queue having a status which includes a queue threshold; a flow history table maintaining the history of flows associated with each said fabric port; and a port selector, said port selector using the flow identifier, the flow history table and said status to forward each of said packets to one of said fabric ports via its associated queue.

2. An apparatus according to claim 1 wherein said fabric ports operate at a lower maximum speed than said network device system ports.

3. An apparatus according to claim 1 wherein said flow identifier distinguishes among flows originating behind a network address translation gateway.

4. An apparatus according to claim 3 wherein said flow identifier is generated using a plurality of header fields belonging to said each packet.

5. An apparatus according to claim 4 wherein said header fields includes at least one of an Internet Protocol address, a Source Port ID, and a Protocol.

6. An apparatus according to claim 1 wherein said port selector comprises: a selection algorithm state machine, said state machine configured to test conditions at said fabric ports and their associated queues, said testing assisting said port selector in determining which of said fabric ports can be selected for said packet.

7. An apparatus according to claim 1 wherein said network device sub-system includes a line card sub-system.

8. An apparatus according to claim 1 wherein said network device includes a service processing card sub-system.

9. An apparatus according to claim 1 wherein said network device is an edge router.

10. An apparatus according to claim 1 wherein said queue threshold is programmable.

11. An apparatus according to claim 1 wherein said flow history table has a programmable depth.

12. An apparatus according to claim 1 wherein said queue counters track the level of utilization of each of said queues.

13. An apparatus according to claim 1 wherein said port selector receives a queue threshold feedback signal from said queues to determine which port's queue buffers are available.

14. An apparatus according to claim 1 wherein flow history table is implemented as a content addressable memory.

15. An apparatus according to claim 14 wherein said content addressable memory is segmented, each segment corresponding to one of said fabric ports, each segment storing the flow history of packets occurring at the switch port to which it corresponds.

16. A method for load balancing packets within a network device, said network device having a plurality of network device sub-systems, each said sub-system having at least one network device system port and a plurality of fabric ports, said fabric ports connecting said each sub-system with others of said sub-systems within said network device, said method comprising: classifying each of said packets according to a network flow to which they belong, all packets of the same network flow classified identically; and selecting one of said fabric ports for sending said classified packet from the network device sub-system containing the one said fabric port, further said fabric ports having a lower maximum operating speed than said at least one network device system port maintaining a flow history table, said flow history table containing a history of previous network flows at each of said fabric ports; and maintaining a plurality of queue counters, each of said queue counters associated with one said queue, said queue counters tracking the space utilized by packets in said associated queue.

17. A method according to claim 16 wherein said considering includes: finding a set of matching fabric ports, any of said fabric ports determined to be matching if that fabric port has previously serviced packets belonging to the same network flow; and finding a set of non-matching switch ports, said non-matching switch ports all those switch ports not found in said set of matching switch ports.

18. A method according to claim 17 wherein said selecting includes: considering one of said matching fabric ports for selection; examining the queue counter corresponding to the queue associated with the considered matching fabric port; repeating the steps of examining and considering until a selection condition becomes true; and setting for said selecting the matching fabric port upon which said selection condition became true.

19. A method according to claim 18 wherein said selection condition is true when the matching fabric port with the largest available space in the queue associated therewith is found among all said queues associated with all said considered matching fabric ports, said selection condition not true if said queue associated therewith has passed a predetermined threshold.

20. A method according to claim 18 wherein if said selection condition does not become true for any of said matching fabric ports, then said selecting further includes: considering one of said non-matching fabric ports for selection; examining the queue counter, corresponding to the queue associated with the considered non-matching fabric port; repeating the steps of examining and considering until a selection condition becomes true, said selection condition becoming true when the non-matching fabric port with the largest available space in the queue associated therewith is found among all said queues associated with all said considered non-matching fabric ports, said selection condition not true if said queue associated therewith has passed a predetermined threshold; and setting for said selecting the non-matching fabric port upon which said selection condition became true.

21. A method according to claim 20 wherein if all ports have reached their threshold, then said classified packet is dropped from passing through said network device.

22. A method according to claim 18 wherein if the set of matching fabric ports is null, then said selecting further includes: considering one of said non-matching fabric ports for selection; examining the queue counter corresponding to the queue associated with the considered non-matching fabric port; repeating the steps of examining and considering until a selection condition becomes true, said selection condition becoming true when the fabric port with the largest available space in the queue associated therewith is found among all said queues associated with all said considered non-matching fabric ports, said selection condition not true if said queue associated therewith has passed a predetermined threshold; and setting for said selecting the non-matching egress fabric port upon which said selection condition became true.

23. A method according to claim 20 wherein if all ports have reached their threshold, then said classified packet is dropped from said network device.

24. A method according to claim 16 wherein said classifying includes: obtaining header fields for said classified packet; and generating a flow identifier from said header fields.

25. A method according to claim 16 wherein maintaining said flow history table includes: adding said flow identifier for said classified packet to the flow history table in the entry corresponding to said selected fabric port.

26. A method according to claim 16 wherein maintaining said queue counters includes: incrementing the queue counter corresponding to the queue of the selected port; and decrementing each queue counter at the line rate of said egress fabric ports for every clock tick.