Local context-based recognition of sketched diagrams

We present a new methodology aimed at the design and implementation of a framework for sketch recognition enabling the recognition and interpretation of diagrams. The diagrams may contain different types of sketched graphic elements such as symbols, connectors, text. Once symbols are distinguished from connectors and identified, the recognition proceeds by identifying the local context of each symbol. This is seen as the symbol interface exposed to the rest of the diagram and includes predefined attachment areas on each symbol. We argue that, in many cases, simple constraints on the local context of each symbol are enough to describe diagram languages defined on those symbols. Further refinement and interpretation of the set of acceptable diagrams is then provided through a visual grammar. We also describe the architecture of the framework and provide sample applications for the domains of flowcharts and binary trees.