Project:

Natural Sketch Recognition in UML Class Diagrams

Sketching is a natural and integral part of software design. Software developers use sketching to aid in the brainstorming of ideas, visualizing programming organization and understanding of requirements. Unfortunately when it comes to coding the system, the drawings are left behind. We see natural sketch recognition as a way to bridge that gap.

We created a natural sketch recognition environment for UML (Unified Modeling Language). Our system differs from graffiti-based approaches to this task, in that it recognizes objects by how they look, not by how they were drawn. Our goal is a system where the user can sketch UML diagrams on a tablet or whiteboard in the same way they would on paper, but the diagrams would then be recognized by the computer.

We have selected UML diagrams because they are a de facto standard for depicting software applications. Within UML we focused on class diagrams, first because of their central role in describing program structure, and second because many of the symbols used in class diagrams are quite similar, and hence they offer an interesting challenge for sketch recognition.



Videos

• Screen video of Tahuti in action (.AVI)
• Movie of Tahuti being used through a projector (.MOV)

Posters

Hammond, Tracy and Davis, Randall (2002). Tahuti: A Geometrical Sketch Recognition System for UML Class Diagrams. AI Lab Visit Weekend

Talks

	Hammond, Tracy and Davis, Randall (2002). Tahuti: A Geometrical Sketch Recognition System for UML Class Diagrams. 2002 AAAI Spring Symposium on Sketch Understanding. Presented 03-24-2002. HTML PPT
	Hammond, Tracy and Gajos, Krzysztof and Davis, Randall and Shrobe, Howard (2002). An Agent-Based System for Capturing and Indexing Software Design Meetings. Proceedings of 2002 International Workshop on Agents in Design (WAID 2002), August 28-30 2002. [PPT] [HTM]

Publications

• Hammond, Tracy and Gajos, Krzysztof and Davis, Randall and Shrobe, Howard (2002). Sketch Recognition in Software Design MIT Artificial Intelligence Laboratory Annual Abstract, September 2002. [PDF] [PS]
• Hammond, Tracy and Gajos, Krzysztof and Davis, Randall and Shrobe, Howard (2002). An Agent-Based System for Capturing and Indexing Software Design Meetings. Proceedings of 2002 International Workshop on Agents in Design (WAID 2002), August 28-30 2002. [DOC] [PPT Talk]
• Hammond, Tracy and Davis, Randall (2002). Tahuti: A Geometrical Sketch Recognition System for UML Class Diagrams. 2002 AAAI Spring Symposium on Sketch Understanding. [PS] , [PDF]
• Hammond, Tracy and Davis, Randall (2001). Tahuti: A Sketch Recognition System for UML Class Diagrams. (Extended Abstract) Accepted for the 2002 AAAI Spring Sumposium on Sketch Understanding. [PS] , [PDF]
• Hammond, Tracy and Oshiro, Kalani and Davis, Randall (2001). Natural Editing and Recognition of UML Class Diagrams MIT Artificial Intelligence Laboratory Annual Abstract, September 2001. [PS] , [PDF]
• Tracy Hammond. Natural Sketch Recognition in UML Class Diagrams. MIT Student Oxygen Workshop 2001, [PS], [PDF]