Method And Tool For Recognizing A Hand-Drawn Table

1. A computerized method of recognizing an input hand-drawn table formed by freeform line objects, said method comprising: transforming each freeform line object of said table into one of a vertical line segment and a horizontal line segment; generating a grid system based on the vertical and horizontal line segments; and converting the generated grid system into a table object, wherein said method further comprises, prior to said transforming, examining each freeform line object to determine if the freeform line-object represents table object content, the transforming, generating and converting being performed only on freeform line objects that do not represent table object content, wherein said examining comprises determining if the freeform line object resembles a curve and if so, designating the freeform line object as representing table object content and wherein said transforming further comprises: comparing each created horizontal line segment with a horizontal line segment list to determine if it overlaps with an existing horizontal line segment; if not, adding the created horizontal line segment to the list and, if so, performing the merging; comparing each created vertical line segment with a vertical line segment list to determine if it overlaps with an existing vertical line segment; and if not, adding the created vertical line segment to the list and, if so, performing the merging and prior to said comparing to determine overlap: examining each horizontal line segment to determine if it is within a threshold vertical distance of an existing horizontal line segment in the horizontal line segment list; if so, performing said comparing to determine overlap and, if not, bypassing the overlap comparing and adding the horizontal line segment to the list; and examining each vertical line segment to determine if it is within a threshold horizontal distance of an existing vertical line segment in the vertical line segment list; and if so, performing said comparing to determine overlap and, if not, bypassing the overlap comparing and adding the vertical line segment to the list.

2. The method of claim 1 wherein said transforming comprises for each freeform line object: examining the orientation of the freeform line object; and creating one of the horizontal and vertical line segment representing the freeform line object based on the orientation thereof.

3. The method of claim 2 wherein said transforming further comprises for each freeform line object: determining the start point and end point of the freeform line object; generating a temporary line segment extending between the determined start point and end point; determining the orientation of said temporary line segment; and based on the determined orientation, creating one of said horizontal line segment and vertical line segment.

4. The method of claim 3 wherein said transforming further comprises: for each created horizontal line segment, determining whether it overlaps with an existing horizontal line segment and, if so, merging the created and existing horizontal line segments; and for each created vertical line segment, determining whether it overlaps with an existing vertical line segment and, if so, merging the created and existing vertical line segments.

5. The method of claim 1 wherein said grid system is generated based on deemed intersections of the horizontal and vertical line segments.

6. The method of claim 5 wherein said grid system generating comprises: selecting pairs of adjacent vertical and adjacent horizontal line segments; and for each pair of adjacent vertical line segments and each pair of adjacent horizontal line segments, determining the intersection points therebetween thereby to yield four comer coordinates of each grid in the grid system.

7. The method of claim 6 wherein said grid system generating comprises: for each pair of adjacent vertical line segments and each pair of adjacent horizontal line segments, arranging the vertical and horizontal line segments into different combinations, each combination comprising one vertical line segment and one horizontal line segment; and for each combination, calculating the intersection point between the vertical and horizontal line segments thereby to calculate said comer coordinates.

8. The method of claim 7 further comprising calculating an effective intersection point for each combination of non-intersecting vertical and horizontal line segments that are within a threshold separation distance.

9. The method of claim 6 wherein said converting comprises using the four comer coordinates of each grid to define comers of a corresponding cell of said table object.

10. The method of claim 9 wherein said grid system generating comprises: for each pair of adjacent vertical line segments and each pair of adjacent horizontal line segments, arranging the vertical and horizontal line segments into different combinations, each combination comprising one vertical line segment and one horizontal line segment; and for each combination, calculating the intersection point between the vertical and horizontal line segments thereby to calculate said comer coordinates.

11. The method of claim 9 wherein said transforming comprises for each freeform line object: examining the orientation of the freeform line object; and creating one of the horizontal and vertical line segment representing the freeform line object based on the orientation thereof.

12. The method of claim 11 wherein said transforming further comprises for each freeform line object: determining the start point and end point of the freeform line object; generating a temporary line segment extending between the determined start point and end point; determining the orientation of said temporary line segment; and based on the determined orientation, creating one of said horizontal line segment and vertical line segment.

13. The method of claim 12 wherein said transforming further comprises: for each created horizontal line segment, determining whether it overlaps with an existing horizontal line segment and, if so, merging the created and existing horizontal line segments; and for each created vertical line segment, determining whether it overlaps with an existing vertical line segment and, if so, merging the created and existing vertical line segments.

14. The method of claim 1 further comprising: detecting input non-freeform line objects that are within the table object; and inserting the detected non-freeform line objects as table object content.

15. The method of claim 14 wherein during said inserting, each detected non-freeform table line object is placed into a corresponding cell of said table object as table cell content.

16. The method of claim 15 wherein said detecting comprises determining non-freeform line objects having designated locations positioned within a cell of the table object.

17. The method of claim 15 wherein said transforming comprises for each freeform line object: examining the orientation of the freeform line object; and creating one of the horizontal and vertical line segment representing the freeform line object based on the orientation thereof.

18. The method of claim 17 wherein said transforming further comprises for each freeform line object: determining the start point and end point of the freeform line object; generating a temporary line segment extending between the determined start point and end point; determining the orientation of said temporary line segment; and based on the determined orientation, creating one of said horizontal line segment and vertical line segment.

19. The method of claim 18 wherein said transforming further comprises: for each created horizontal line segment, determining whether it overlaps with an existing horizontal line segment and, if so, merging the created and existing horizontal line segments; and for each created vertical line segment, determining whether it overlaps with an existing vertical line segment and, if so, merging the created and existing vertical line segments.

20. The method of claim 19 wherein said grid system is generated based on deemed intersections of the horizontal and vertical line segments.

21. The method of claim 20 wherein said grid system generating comprises: selecting pairs of adjacent vertical and adjacent horizontal line segments; and for each pair of adjacent vertical line segments and each pair of adjacent horizontal line segments, determining the intersection points therebetween thereby to yield four comer coordinates of each grid in the grid system.

22. The method of claim 21 wherein said grid system generating comprises: for each pair of adjacent vertical line segments and each pair of adjacent horizontal line segments, arranging the vertical and horizontal line segments into different combinations, each combination comprising one vertical line segment and one horizontal line segment; and for each combination, calculating the intersection point between the vertical and horizontal line segments thereby to calculate said comer coordinates.

23. The method of claim 15 further comprising processing the table object based on the table cell content placed in the table object.

24. The method of claim 1 wherein said examining further comprises, if the freeform line object does not resemble a curve, determining if the freeform line object resembles a letter, if so determining if the letter is a letter that resembles the shape of a table outline and if not, designating the freeform line object as representing table object content.

25. The method of claim 24 , wherein if the letter resembles the shape of a table outline, examining the slopes of line segments making up the freeform line object to determine if each line segment resembles either a vertical line or a horizontal line, and if not, designating the freeform line object as representing table object content.

26. The method of claim 25 wherein if the freeform line object does not resemble a letter, determining if the freeform line object represents an oblique line and if so, designating the freeform line object as representing table object content.

27. The method of claim 26 further comprising identifying each transformed freeform line object not used to generate said grid system and processing the freeform line object to determine if the freeform line object represents one of a table outline element, table object content and object content external to said table object.

28. The method of claim 25 further comprising identifying each transformed freeform line object not used to generate said grid system and processing the freeform line object to determine if the freeform line object represents one of a table outline element, table object content and object content external to said table object.

29. The method of claim 25 wherein if the freeform line object does not resemble a letter, determining if the freeform line object represents an oblique line and if so, identifying the freeform line object as a table cell partition candidate.

30. The method of claim 29 further comprising after said converting processing each table cell partition candidate to verify the same and sub-dividing cells of said table object using verified table cell partitions.

31. A non-transitory computer readable medium embodying computer program code for recognizing an input hand-drawn table formed by freeform line objects, said computer program code comprising: program code for transforming each freeform line object of said table into one of a vertical line segment and a horizontal line segment; program code for generating a grid system based on the vertical and horizontal line segments; program code for converting the generated grid system into a table object; and program code for, prior to said transforming, examining each freeform line object to determine if the freeform line-object represents table object content, the transforming, generating and converting being performed only on freeform line objects that do not represent table object content, wherein said examining comprises determining if the freeform line object resembles a curve and if so, designating the freeform line object as representing table object content and wherein said transforming further comprises: comparing each created horizontal line segment with a horizontal line segment list to determine if it overlaps with an existing horizontal line segment; if not, adding the created horizontal line segment to the list and, if so, performing the merging; comparing each created vertical line segment with a vertical line segment list to determine if it overlaps with an existing vertical line segment; and if not, adding the created vertical line segment to the list and, if so, performing the merging and prior to said comparing to determine overlap: examining each horizontal line segment to determine if it is within a threshold vertical distance of an existing horizontal line segment in the horizontal line segment list; if so, performing said comparing to determine overlap and, if not, bypassing the overlap comparing and adding the horizontal line segment to the list; and examining each vertical line segment to determine if it is within a threshold horizontal distance of an existing vertical line segment in the vertical line segment list; and if so, performing said comparing to determine overlap and, if not, bypassing the overlap comparing and adding the vertical line segment to the list.

32. An interactive input system comprising: a coordinate input device having a display surface on which an image is presented, said coordinate input device being responsive to user input; and processing structure communicating with said coordinate input device and updating the image presented on said display surface to reflect user input, said processing structure executing a table recognition procedure when invoked in response to user input to recognize a hand-drawn table formed by freeform line objects entered on said coordinate input device and convert the recognized hand-drawn table into a table object, wherein said table recognition procedure comprises computer program code comprising: program code for transforming each freeform line object of said table into one of a vertical line segment and a horizontal line segment; program code for generating a grid system based on the vertical and horizontal line segments; program code for converting the generated grid system into a table object; and program code for, prior to said transforming, examining each freeform line object to determine if the freeform line-object represents table object content, the transforming, generating and converting being performed only on freeform line objects that do not represent table object content, wherein during execution of said table recognition procedure, said processing structure detects other displayed objects that overlap with said hand-drawn table and places the detected other objects as table cell content in cells of said table object and wherein said transforming further comprises: comparing each created horizontal line segment with a horizontal line segment list to determine if it overlaps with an existing horizontal line segment; if not, adding the created horizontal line segment to the list and, if so, performing the merging; comparing each created vertical line segment with a vertical line segment list to determine if it overlaps with an existing vertical line segment; and if not, adding the created vertical line segment to the list and, if so, performing the merging and prior to said comparing to determine overlap: examining each horizontal line segment to determine if it is within a threshold vertical distance of an existing horizontal line segment in the horizontal line segment list; if so, performing said comparing to determine overlap and, if not, bypassing the overlap comparing and adding the horizontal line segment to the list; and examining each vertical line segment to determine if it is within a threshold horizontal distance of an existing vertical line segment in the vertical line segment list; and if so, performing said comparing to determine overlap and, if not, bypassing the overlap comparing and adding the vertical line segment to the list.

33. An interactive input system comprising: a coordinate input device having a display surface on which an image is presented, said coordinate input device being responsive to user input; and processing structure communicating with said coordinate input device and updating the image presented on said display surface to reflect user input, said processing structure executing a table recognition procedure when invoked in response to user input to recognize a hand-drawn table formed by freeform line objects entered on said coordinate input device and convert the recognized hand-drawn table into a table object, wherein said table recognition procedure comprises computer program code comprising: program code for transforming each freeform line object of said table into one of a vertical line segment and a horizontal line segment; program code for generating a grid system based on the vertical and horizontal line segments; program code for converting the generated grid system into a table object; and program code for, prior to said transforming, examining each freeform line object to determine if the freeform line-object represents table object content, the transforming, generating and converting being performed only on freeform line objects that do not represent table object content, wherein during said hand-drawn table recognition, said processing structure transforms each freeform line object forming said hand-drawn table into one of a vertical line segment and a horizontal line segment, generates a grid system based on the vertical and horizontal line segments and converts the generated grid system into said table object and wherein said transforming further comprises: comparing each created horizontal line segment with a horizontal line segment list to determine if it overlaps with an existing horizontal line segment; if not, adding the created horizontal line segment to the list and, if so, performing the merging; comparing each created vertical line segment with a vertical line segment list to determine if it overlaps with an existing vertical line segment; and if not, adding the created vertical line segment to the list and, if so, performing the merging and prior to said comparing to determine overlap: examining each horizontal line segment to determine if it is within a threshold vertical distance of an existing horizontal line segment in the horizontal line segment list; if so, performing said comparing to determine overlap and, if not, bypassing the overlap comparing and adding the horizontal line segment to the list; and examining each vertical line segment to determine if it is within a threshold horizontal distance of an existing vertical line segment in the vertical line segment list; and if so, performing said comparing to determine overlap and, if not, bypassing the overlap comparing and adding the vertical line segment to the list.

34. A computerized method of recognizing an input hand-drawn table formed by freeform line objects, said method comprising: transforming each freeform line object of said table into one of a vertical line segment and a horizontal line segment; generating a grid system based on the vertical and horizontal line segments; converting the generated grid system into a table object, wherein said method further comprises, prior to said transforming, examining each freeform line object to determine if the freeform line-object represents table object content, the transforming, generating and converting being performed only on freeform line objects that do not represent table object content; detecting input non-freeform line objects that are within the table object; and inserting the detected non-freeform line objects as table object content and wherein said transforming further comprises: comparing each created horizontal line segment with a horizontal line segment list to determine if it overlaps with an existing horizontal line segment; if not, adding the created horizontal line segment to the list and, if so, performing the merging; comparing each created vertical line segment with a vertical line segment list to determine if it overlaps with an existing vertical line segment; and if not, adding the created vertical line segment to the list and, if so, performing the merging and prior to said comparing to determine overlap: examining each horizontal line segment to determine if it is within a threshold vertical distance of an existing horizontal line segment in the horizontal line segment list; if so, performing said comparing to determine overlap and, if not, bypassing the overlap comparing and adding the horizontal line segment to the list; and examining each vertical line segment to determine if it is within a threshold horizontal distance of an existing vertical line segment in the vertical line segment list; and if so, performing said comparing to determine overlap and, if not, bypassing the overlap comparing and adding the vertical line segment to the list.

35. The method of claim 34 wherein during said inserting, each detected non-freeform line object is placed into a corresponding cell of said table object as table cell content.

36. The method of claim 35 wherein said detecting comprises determining non-freeform line objects having designated locations positioned within a cell of the table object.

37. The method of claim 35 wherein said transforming comprises for each freeform line object: examining the orientation of the freeform line object; and creating one of the horizontal and vertical line segment representing the freeform line object based on the orientation thereof.

38. The method of claim 37 wherein said transforming further comprises for each freeform line object: determining the start point and end point of the freeform line object; generating a temporary line segment extending between the determined start point and end point; determining the orientation of said temporary line segment; and based on the determined orientation, creating one of said horizontal line segment and vertical line segment.

39. The method of claim 38 wherein said transforming further comprises: for each created horizontal line segment, determining whether it overlaps with an existing horizontal line segment and, if so, merging the created and existing horizontal line segments; and for each created vertical line segment, determining whether it overlaps with an existing vertical line segment and, if so, merging the created and existing vertical line segments.

40. The method of claim 39 wherein said grid system is generated based on deemed intersections of the horizontal and vertical line segments.

41. The method of claim 40 wherein said grid system generating comprises: selecting pairs of adjacent vertical and adjacent horizontal line segments; and for each pair of adjacent vertical line segments and each pair of adjacent horizontal line segments, determining the intersection points therebetween thereby to yield four comer coordinates of each grid in the grid system.

42. The method of claim 41 wherein said grid system generating comprises: for each pair of adjacent vertical line segments and each pair of adjacent horizontal line segments, arranging the vertical and horizontal line segments into different combinations, each combination comprising one vertical line segment and one horizontal line segment; and for each combination, calculating the intersection point between the vertical and horizontal line segments thereby to calculate said comer coordinates.

43. The method of claim 35 further comprising processing the table object based on the table cell content placed in the table object.

44. The method of claim 43 wherein processing the table object comprises: recognizing the content in a first table cell as a predetermined processing indicator; processing the content in one or more table cells differing from said first table cell according to the recognized processing indicator; and replacing the content in said first table cell with the result of the processing.

45. The method of claim 44 wherein the content in the first table cell is a hand-drawn object.

46. The method of claim 45 wherein said processing indicator is a mathematical plus sign and wherein the result of the processing is a summation.

47. A computerized method of recognizing an input hand-drawn table formed by freeform line objects, said method comprising: transforming each freeform line object of said table into one of a vertical line segment and a horizontal line segment; generating a grid system based on the vertical and horizontal line segments; and converting the generated grid system into a table object, wherein said method further comprises, prior to said transforming, examining each freeform line object to determine if the freeform line-object represents table object content, the transforming, generating and converting being performed only on freeform line objects that do not represent table object content, wherein said grid system is generated based on deemed intersections of the horizontal and vertical line segments and wherein said grid system generating comprises: selecting pairs of adjacent vertical and adjacent horizontal line segments; for each pair of adjacent vertical line segments and each pair of adjacent horizontal line segments, arranging the vertical and horizontal line segments into different combinations, each combination comprising one vertical line segment and one horizontal line segment; for each combination, calculating the intersection point between the vertical and horizontal line segments thereby to calculate comer coordinates of each grid in the grid system; and calculating an effective intersection point for each combination of non-intersecting vertical and horizontal line segments that are within a threshold separation distance.

48. A computerized method of recognizing an input hand-drawn table formed by freeform line objects, said method comprising: transforming each freeform line object of said table into one of a vertical line segment and a horizontal line segment; generating a grid system based on the vertical and horizontal line segments; and converting the generated grid system into a table object, wherein said method further comprises, prior to said transforming, examining each freeform line object to determine if the freeform line-object represents table object content, the transforming, generating and converting being performed only on freeform line objects that do not represent table object content, wherein said transforming comprises for each freeform line object: determining the start point and end point of the freeform line object; generating a temporary line segment extending between the determined start point and end point; determining the orientation of said temporary line segment; and transforming further comprises: comparing each created horizontal line segment with a horizontal line segment list to determine if it overlaps with an existing horizontal line segment; if not, adding the created horizontal line segment to the list and, if so, performing the merging; comparing each created vertical line segment with a vertical line segment list to determine if it overlaps with an existing vertical line segment; and if not, adding the created vertical line segment to the list and, if so, performing the merging, and based on the determined orientation, creating one of said horizontal line segment and vertical line segment; for each created horizontal line segment, determining whether it overlaps with an existing horizontal line segment and, if so, merging the created and existing horizontal line segments; and for each created vertical line segment, determining whether it overlaps with an existing vertical line segment and, if so, merging the created and existing vertical line segments.

49. The method of claim 48 wherein said transforming further comprises, prior to said comparing to determine overlap: examining each horizontal line segment to determine if it is within a threshold vertical distance of an existing horizontal line segment in the horizontal line segment list; if so, performing said comparing to determine overlap and, if not, bypassing the overlap comparing and adding the horizontal line segment to the list; and examining each vertical line segment to determine if it is within a threshold horizontal distance of an existing vertical line segment in the vertical line segment list; and if so, performing said comparing to determine overlap and, if not, bypassing the overlap comparing and adding the vertical line segment to the list.