Method and Apparatus for Recognising Music Symbols

1. A method, implemented by a music symbol recognition apparatus for recognising music symbols based on handwritten music notations, said method comprising: detecting handwritten music notations; pre-segmenting said handwritten music notations into a plurality of elementary ink segments; grouping the elementary ink segments into graphical objects based on spatial relationships between elementary ink segments, wherein each elementary ink segment belongs to one or more of said graphical objects; determining for each graphical object at least one music symbol candidate, in association with an assigned symbol cost, the symbol cost representing the probability of said graphical object belonging to a predetermined class of said music symbol candidate, said determining being based on graphical features extracted from said graphical object; and parsing the music symbol candidates, wherein said parsing comprises: forming one or more graphs by applying at least one of a predetermined set of grammar rules to said music symbol candidates, wherein each graph comprises at least one non-terminal node corresponding to a grammar rule applied to a set of at least one descendant node, and wherein each descendant node is either a terminal node corresponding to a music symbol candidate or a non-terminal-node corresponding to a grammar rule applied to at least one other descendant node; associating each grammar rule applied to at least two descendant nodes with a spatial cost representative of the pertinence of said applied grammar rule based on the spatial relationships between the graphical objects of said at least two descendant nodes; and selecting at least one said graph as the most representative graph of the handwritten music notations based on the symbol costs associated with each music symbol candidate and the spatial costs associated with each applied grammar rule.

2. The method according to claim 1 , wherein forming one or more graphs comprises: attempting recursively to apply each of the set of grammar rules to said music symbol candidates.

3. The method according to claim 1 , wherein parsing the music symbol candidates comprises: calculating for each graph a total cost, taking into account each symbol cost assigned to the music symbol candidates of said graphs and each spatial cost associated with the at least one grammar rule applied in said graph.

4. The method according to claim 3 , wherein selecting at least one said graph comprises: selecting at least one said graph based on the total costs obtained for each graph.

5. The method according to claim 3 , wherein the total cost for each graph is obtained by summing at least each spatial cost and symbol cost of each graph.

6. The method according to claim 3 , wherein selecting at least one said graph comprises: determining each possible graph representing the detected handwritten music notations and choosing the graph having the lowest total cost.

7. The method according to claim 1 , further comprising: displaying on a display of said music symbol recognition apparatus the music symbol candidates of the at least one selected graph.

8. The method according to claim 7 , wherein each music symbol candidate which is displayed replaces the graphical objects corresponding to each music symbol candidate that are present on said display.

9. The method according to claim 1 , further comprising: normalising said detected handwritten music notations prior to said pre-segmenting.

10. The method according to claim 1 , wherein said determining for each graphical object at least one music symbol candidate is performed by a neural network.

11. The method according to claim 1 , wherein each grammar rule defines, when applicable: a predetermined association of a music symbol or a group of music symbols with at least a non-terminal node, and a value of the spatial cost based on the spatial relationships of the graphical objects corresponding to at least two descendant nodes, when there are at least two descendant nodes in said predetermined association.

12. The method according to claim 1 , further comprising: producing a parse tree based on each graph selected as the most representative graph of the handwritten music notations that were detected.

13. The method according to claim 1 , wherein determining for each graphical object at least one music symbol candidate comprises: extracting at least one of a static graphical feature and a dynamic graphical feature of said graphical objects, said determining being based on the result of said extracting.

14. The method according to claim 1 , wherein detecting handwritten music notations comprises: detecting said handwritten music symbols which are inputted by a user on an input surface of said music symbol recognition apparatus.

15. A non-transitory computer-readable medium including instructions, that when executed by a processor, perform a method comprising: detecting handwritten music notations; pre-segmenting said handwritten music notations into a plurality of elementary ink segments; grouping the elementary ink segments into graphical objects based on spatial relationships between elementary ink segments, wherein each elementary ink segment belongs to one or more of said graphical objects; determining for each graphical object at least one music symbol candidate, in association with an assigned symbol cost, the symbol cost representing the probability of said graphical object belonging to a predetermined class of said music symbol candidate, said determining being based on graphical features extracted from said graphical object; and parsing the music symbol candidates, wherein said parsing comprises: forming one or more graphs by applying at least one of a predetermined set of grammar rules to said music symbol candidates, wherein each graph comprises at least one non-terminal node corresponding to a grammar rule applied to a set of at least one descendant node, and wherein each descendant node is either a terminal node corresponding to a music symbol candidate or a non-terminal-node corresponding to a grammar rule applied to at least one other descendant node; associating each grammar rule applied to at least two descendant nodes with a spatial cost representative of the pertinence of said applied grammar rule based on the spatial relationships between the graphical objects of said at least two descendant nodes; and selecting at least one said graph as the most representative graph of the handwritten music notations based on the symbol costs associated with each music symbol candidate and the spatial costs associated with each applied grammar rule.

16. A music symbol recognition apparatus for recognising music symbols based on handwritten music notations, the apparatus comprising: a detecting unit for detecting handwritten music notations; a pre-segmenting unit for pre-segmenting said handwritten music notations into a plurality of elementary ink segments; a grouping unit for grouping the elementary ink segments into graphical objects based on spatial relationships between elementary ink segments, wherein each elementary ink segment belongs to one or more of said graphical objects; a determining unit for determining for each graphical object at least one music symbol candidate, in association with an assigned symbol cost, the symbol cost representing the probability of said graphical object belonging to a predetermined class of said music symbol candidate, said determination being based on graphical features extracted from said graphical object; and a parsing unit for parsing the music symbol candidates, wherein said parsing unit comprises: a forming unit for forming one or more graphs by applying at least one of a predetermined set of grammar rules to said music symbol candidates, wherein each graph comprises at least one non-terminal node corresponding to a grammar rule applied to a set of at least one descendant node, wherein each descendant node is either a terminal node corresponding to a music symbol candidate or a non-terminal-node corresponding to a grammar rule applied to at least one other descendant node; an associating unit for associating each applied grammar rule applied to at least two descendant nodes with a spatial cost representative of the pertinence of said applied grammar rule based on the spatial relationships between the graphical objects of said at least two descendant nodes; and a selecting unit for selecting at least one said graph as the most representative graph of the handwritten music notations based on the symbol costs associated with each music symbol candidate and the spatial costs associated with each applied grammar rule.

17. The apparatus according to claim 16 , further comprising: an input surface, wherein said detecting unit is configured to detect the handwritten music notations inputted by a user on said input surface.

18. The apparatus according to claim 16 , wherein said determining unit comprises a neural network.