Methods and apparatuses for hierarchical partitioning of operators of a neural network for execution on an acceleration engine are provided. Neural networks are built in machine learning frameworks using neural network operators. The neural network operators are compiled into executable code for the acceleration engine. Development of new framework-level operators can exceed the capability to map the newly developed framework-level operators onto the acceleration engine. To enable neural networks to be executed on an acceleration engine, hierarchical partitioning can be used to partition the operators of the neural network. The hierarchical partitioning can identify operators that are supported by a compiler for execution on the acceleration engine, operators to be compiled for execution on a host processor, and operators to be executed on the machine learning framework.
Legal claims defining the scope of protection, as filed with the USPTO.
2. The method of claim 1, further comprising compiling the neural network operators of the third set of neural network operators having parameters that are supported by the acceleration engine for execution on the acceleration engine.
3. The method of claim 1, further comprising compiling the fourth set of neural network operators for execution on the host processor.
6. The method of claim 5, further comprising partitioning the neural network operators that are not on the list of neural network operators that are supported by the compiler for execution on the machine learning framework.
15. The system of claim 10, wherein the acceleration engine is configured to execute the first compiled model.
16. The system of claim 10, wherein the set of processors is configured to execute the second compiled model.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
November 27, 2019
December 31, 2024
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.