Method, Apparatus, and Computer-Readable Medium for Parallelization of a Computer Program on a Plurality of Computing Cores

1. A method executed by one or more computing devices for parallelization of a computer program on a plurality of computing cores, the method comprising: receiving, by at least one of the one or more computing devices, the computer program comprising a plurality of commands; decomposing, by at least one of the one or more computing devices, the plurality of commands into a plurality of node networks, wherein each node network corresponds to a command in the plurality of commands and includes one or more nodes corresponding to execution dependencies of the command; mapping, by at least one of the one or more computing devices, the plurality of node networks to a plurality of systolic arrays, wherein each systolic array comprises a plurality of cells and wherein each non-data node in each node network is mapped to a cell in the plurality of cells; and mapping, by at least one of the one or more computing devices, each cell in each systolic array to a computing core in the plurality of computing cores.

2. The method of claim 1 , wherein decomposing the plurality of commands into a plurality of node networks comprises, for each command: parsing the command using a lexical analyzer to identify one or more components of the command; storing the command as header node in a node network corresponding to the command; and storing the one or more components as nodes under the header node.

3. The method of claim 2 , wherein the lexical analyzer is implemented in hardware by one or more systolic arrays.

4. The method of claim 2 , wherein the lexical analyzer is generated based at least in part on a language definition file corresponding to a programming language of the computer program.

5. The method of claim 2 , wherein the one or more components comprise one or more of a sub-command and a data value.

6. The method of claim 5 , wherein decomposing the plurality of commands into a plurality of node networks further comprises, for each command: identifying one or more sub-commands in the one or more components using the lexical analyzer; parsing the one or more sub-commands using the lexical analyzer to identify one or more sub-components of the one or more sub-commands; and storing the one or more sub-components as nodes under one or more nodes corresponding to the one or more sub-commands.

7. The method of claim 1 , wherein mapping the plurality of node networks to a plurality of systolic arrays comprises, for each node network: mapping a plurality of non-data nodes in the node network to a plurality of cells in a systolic array; connecting the plurality of cells in the systolic array based at least in part on connections between the nodes in the node network; and connecting one or more cells in the plurality of cells to one or more other systolic arrays.

8. The method of claim 7 , wherein at least one cell in the plurality of cells corresponds to a command or sub-command and wherein connecting the plurality of cells in the systolic array comprises: connecting the at least one cell corresponding to the command or sub-command to one or more other cells in the plurality of cells based at least in part on a determination that output of the one or more other cells is input to the command or sub-command.

9. The method of claim 8 , wherein the determination that the output of one or more other nodes corresponding to the one or more other cells is input to the command or sub-command is made based on a language definition file corresponding to a programming language of the computer program.

10. The method of claim 7 , wherein mapping the plurality of non-data nodes in the node network to a plurality of cells in a systolic array comprises: adding one or more of an input port and an output port to at least one cell in the plurality of cells.

11. The method of claim 10 , wherein the one or more of an input port and an output port is added based at least in part on a language definition file corresponding to a programming language of the computer program.

12. The method of claim 7 , wherein connecting one or more cells in the plurality of cells to one or more other systolic arrays comprises: identifying each cell in the plurality of cells in the systolic array which corresponds to a sub-command; and connecting each identified cell to another other systolic array corresponding to the sub-command for that identified cell.

13. The method of claim 12 , wherein connecting one or more cells in the plurality of cells to one or more other systolic arrays further comprises: recursively connecting all cells corresponding to sub-commands in the one or more other systolic arrays to additional systolic arrays corresponding to the sub-commands until no unconnected cells corresponding to sub-commands remain.

14. The method of claim 1 , wherein mapping each cell in each systolic array to a computing core in a plurality of computing cores comprises: traversing the plurality of cells in each of the plurality of systolic arrays according to an execution order of the plurality of cells and the plurality of systolic arrays; and mapping each traversed cell to a computing core in the plurality of computing cores, wherein each traversed cell is mapped in the order of traversal.

15. The method of claim 14 , wherein the execution order comprises a critical path.

16. The method of claim 1 , wherein each cell in each systolic array is mapped to the corresponding computing core based at least in part on an order of execution of the plurality of cells in the systolic array and an order of execution of the systolic array in the plurality of systolic arrays.

17. The method of claim 1 , wherein the plurality of computing cores comprise one or more field-programmable gate arrays (FPGAs) and wherein mapping each cell in each systolic array to a computing core in the plurality of computing cores comprises: configuring an FPGA in the one or more of FPGAs to correspond to logic of the cell.

18. An apparatus for parallelization of a computer program on a plurality of computing cores, the apparatus comprising: one or more processors; and one or more memories operatively coupled to at least one of the one or more processors and having instructions stored thereon that, when executed by at least one of the one or more processors, cause at least one of the one or more processors to: receive the computer program comprising a plurality of commands; decompose the plurality of commands into a plurality of node networks, wherein each node network corresponds to a command in the plurality of commands and includes one or more nodes corresponding to execution dependencies of the command; map the plurality of node networks to a plurality of systolic arrays, wherein each systolic array comprises a plurality of cells and wherein each non-data node in each node network is mapped to a cell in the plurality of cells; and map each cell in each systolic array to a computing core in the plurality of computing cores.

19. The apparatus of claim 18 , wherein the instructions that, when executed by at least one of the one or more processors, cause at least one of the one or more processors to decompose the plurality of commands into a plurality of node networks further cause at least one of the one or more processors to, for each command: parse the command using a lexical analyzer to identify one or more components of the command; store the command as header node in a node network corresponding to the command; and store the one or more components as nodes under the header node.

20. The apparatus of claim 19 , wherein the lexical analyzer is implemented in hardware by one or more systolic arrays.

21. The apparatus of claim 19 , wherein the lexical analyzer is generated based at least in part on a language definition file corresponding to a programming language of the computer program.

22. The apparatus of claim 19 , wherein the one or more components comprise one or more of a sub-command and a data value.

23. The apparatus of claim 22 , wherein the instructions that, when executed by at least one of the one or more processors, cause at least one of the one or more processors to decompose the plurality of commands into a plurality of node networks further cause at least one of the one or more processors to, for each command: identify one or more sub-commands in the one or more components using the lexical analyzer; parse the one or more sub-commands using the lexical analyzer to identify one or more sub-components of the one or more sub-commands; and store the one or more sub-components as nodes under one or more nodes corresponding to the one or more sub-commands.

24. The apparatus of claim 18 , wherein the instructions that, when executed by at least one of the one or more processors, cause at least one of the one or more processors to map the plurality of node networks to a plurality of systolic arrays further cause at least one of the one or more processors to, for each node network: map a plurality of non-data nodes in the node network to a plurality of cells in a systolic array; connect the plurality of cells in the systolic array based at least in part on connections between the nodes in the node network; and connect one or more cells in the plurality of cells to one or more other systolic arrays.

25. The apparatus of claim 24 , wherein at least one cell in the plurality of cells corresponds to a command or sub-command and wherein the instructions that, when executed by at least one of the one or more processors, cause at least one of the one or more processors to connect the plurality of cells in the systolic array further cause at least one of the one or more processors to: connect the at least one cell corresponding to the command or sub-command to one or more other cells in the plurality of cells based at least in part on a determination that output of the one or more other cells is input to the command or sub-command.

26. The apparatus of claim 25 , wherein the determination that the output of one or more other nodes corresponding to the one or more other cells is input to the command or sub-command is made based on a language definition file corresponding to a programming language of the computer program.

27. The apparatus of claim 24 , wherein the instructions that, when executed by at least one of the one or more processors, cause at least one of the one or more processors to map the plurality of non-data nodes in the node network to a plurality of cells in a systolic array further cause at least one of the one or more processors to: add one or more of an input port and an output port to at least one cell in the plurality of cells.

28. The apparatus of claim 27 , wherein the one or more of an input port and an output port is added based at least in part on a language definition file corresponding to a programming language of the computer program.

29. The apparatus of claim 24 , wherein the instructions that, when executed by at least one of the one or more processors, cause at least one of the one or more processors to connect one or more cells in the plurality of cells to one or more other systolic arrays further cause at least one of the one or more processors to: identify each cell in the plurality of cells in the systolic array which corresponds to a sub-command; and connect each identified cell to another other systolic array corresponding to the sub-command for that identified cell.

30. The apparatus of claim 29 , wherein the instructions that, when executed by at least one of the one or more processors, cause at least one of the one or more processors to connect one or more cells in the plurality of cells to one or more other systolic arrays further cause at least one of the one or more processors to: recursively connect all cells corresponding to sub-commands in the one or more other systolic arrays to additional systolic arrays corresponding to the sub-commands until no unconnected cells corresponding to sub-commands remain.

31. The apparatus of claim 18 , wherein the instructions that, when executed by at least one of the one or more processors, cause at least one of the one or more processors to map each cell in each systolic array to a computing core in a plurality of computing cores further cause at least one of the one or more processors to: traverse the plurality of cells in each of the plurality of systolic arrays according to an execution order of the plurality of cells and the plurality of systolic arrays; and map each traversed cell to a computing core in the plurality of computing cores, wherein each traversed cell is mapped in the order of traversal.

32. The apparatus of claim 31 , wherein the execution order comprises a critical path.

33. The apparatus of claim 18 , wherein each cell in each systolic array is mapped to the corresponding computing core based at least in part on an order of execution of the plurality of cells in the systolic array and an order of execution of the systolic array in the plurality of systolic arrays.

34. The apparatus of claim 18 , wherein the plurality of computing cores comprise one or more field-programmable gate arrays (FPGAs) and wherein the instructions that, when executed by at least one of the one or more processors, cause at least one of the one or more processors to map each cell in each systolic array to a computing core in the plurality of computing cores further cause at least one of the one or more processors to: configure an FPGA in the one or more of FPGAs to correspond to logic of the cell.

35. At least one non-transitory computer-readable medium storing computer-readable instructions that, when executed by one or more computing devices, cause at least one of the one or more computing devices to: receive a computer program comprising a plurality of commands; decompose the plurality of commands into a plurality of node networks, wherein each node network corresponds to a command in the plurality of commands and includes one or more nodes corresponding to execution dependencies of the command; map the plurality of node networks to a plurality of systolic arrays, wherein each systolic array comprises a plurality of cells and wherein each non-data node in each node network is mapped to a cell in the plurality of cells; and map each cell in each systolic array to a computing core in the plurality of computing cores.

36. The at least one non-transitory computer-readable medium of claim 35 , wherein the instructions that, when executed by at least one of the one or more computing devices, cause at least one of the one or more computing devices to decompose the plurality of commands into a plurality of node networks further cause at least one of the one or more computing devices to, for each command: parse the command using a lexical analyzer to identify one or more components of the command; store the command as header node in a node network corresponding to the command; and store the one or more components as nodes under the header node.

37. The at least one non-transitory computer-readable medium of claim 36 , wherein the lexical analyzer is implemented in hardware by one or more systolic arrays.

38. The at least one non-transitory computer-readable medium of claim 36 , wherein the lexical analyzer is generated based at least in part on a language definition file corresponding to a programming language of the computer program.

39. The at least one non-transitory computer-readable medium of claim 36 , wherein the one or more components comprise one or more of a sub-command and a data value.

40. The at least one non-transitory computer-readable medium of claim 39 , wherein the instructions that, when executed by at least one of the one or more computing devices, cause at least one of the one or more computing devices to decompose the plurality of commands into a plurality of node networks further cause at least one of the one or more computing devices to, for each command: identify one or more sub-commands in the one or more components using the lexical analyzer; parse the one or more sub-commands using the lexical analyzer to identify one or more sub-components of the one or more sub-commands; and store the one or more sub-components as nodes under one or more nodes corresponding to the one or more sub-commands.

41. The at least one non-transitory computer-readable medium of claim 35 , wherein the instructions that, when executed by at least one of the one or more computing devices, cause at least one of the one or more computing devices to map the plurality of node networks to a plurality of systolic arrays further cause at least one of the one or more computing devices to, for each node network: map a plurality of non-data nodes in the node network to a plurality of cells in a systolic array; connect the plurality of cells in the systolic array based at least in part on connections between the nodes in the node network; and connect one or more cells in the plurality of cells to one or more other systolic arrays.

42. The at least one non-transitory computer-readable medium of claim 41 , wherein at least one cell in the plurality of cells corresponds to a command or sub-command and wherein the instructions that, when executed by at least one of the one or more computing devices, cause at least one of the one or more computing devices to connect the plurality of cells in the systolic array further cause at least one of the one or more computing devices to: connect the at least one cell corresponding to the command or sub-command to one or more other cells in the plurality of cells based at least in part on a determination that output of the one or more other cells is input to the command or sub-command.

43. The at least one non-transitory computer-readable medium of claim 42 , wherein the determination that the output of one or more other nodes corresponding to the one or more other cells is input to the command or sub-command is made based on a language definition file corresponding to a programming language of the computer program.

44. The at least one non-transitory computer-readable medium of claim 41 , wherein the instructions that, when executed by at least one of the one or more computing devices, cause at least one of the one or more computing devices to map the plurality of non-data nodes in the node network to a plurality of cells in a systolic array further cause at least one of the one or more computing devices to: add one or more of an input port and an output port to at least one cell in the plurality of cells.

45. The at least one non-transitory computer-readable medium of claim 44 , wherein the one or more of an input port and an output port is added based at least in part on a language definition file corresponding to a programming language of the computer program.

46. The at least one non-transitory computer-readable medium of claim 41 , wherein the instructions that, when executed by at least one of the one or more computing devices, cause at least one of the one or more computing devices to connect one or more cells in the plurality of cells to one or more other systolic arrays further cause at least one of the one or more computing devices to: identify each cell in the plurality of cells in the systolic array which corresponds to a sub-command; and connect each identified cell to another other systolic array corresponding to the sub-command for that identified cell.

47. The at least one non-transitory computer-readable medium of claim 46 , wherein the instructions that, when executed by at least one of the one or more computing devices, cause at least one of the one or more computing devices to connect one or more cells in the plurality of cells to one or more other systolic arrays further cause at least one of the one or more computing devices to: recursively connect all cells corresponding to sub-commands in the one or more other systolic arrays to additional systolic arrays corresponding to the sub-commands until no unconnected cells corresponding to sub-commands remain.

48. The at least one non-transitory computer-readable medium of claim 35 , wherein the instructions that, when executed by at least one of the one or more computing devices, cause at least one of the one or more computing devices to map each cell in each systolic array to a computing core in a plurality of computing cores further cause at least one of the one or more computing devices to: traverse the plurality of cells in each of the plurality of systolic arrays according to an execution order of the plurality of cells and the plurality of systolic arrays; and map each traversed cell to a computing core in the plurality of computing cores, wherein each traversed cell is mapped in the order of traversal.

49. The at least one non-transitory computer-readable medium of claim 48 , wherein the execution order comprises a critical path.

50. The at least one non-transitory computer-readable medium of claim 35 , wherein each cell in each systolic array is mapped to the corresponding computing core based at least in part on an order of execution of the plurality of cells in the systolic array and an order of execution of the systolic array in the plurality of systolic arrays.

51. The at least one non-transitory computer-readable medium of claim 35 , wherein the plurality of computing cores comprise one or more field-programmable gate arrays (FPGAs) and wherein the instructions that, when executed by at least one of the one or more computing devices, cause at least one of the one or more computing devices to map each cell in each systolic array to a computing core in the plurality of computing cores further cause at least one of the one or more computing devices to: configure an FPGA in the one or more of FPGAs to correspond to logic of the cell.