     
|
Publications
Press CoveragePC World: Microsoft Looks in the Crystal Ball: Research chief demonstrates Magic Paper, virtual classrooms, and other projects. October 2003 CNN: Breathing life into messy sketches October 2003 New Scientist: Smart software makes sense of rough sketches September 2003 Mass HighTech: MITs tablet tech gets a look-see from Microsoft July 2003 Journal Articles2008
Sketch recognition in interspersed drawings using time-based graphical models. Computers and Graphics 32(5), pp.500-510. October 2008.
2007
Book Review: Gesture in Human-Computer Interaction and Simulation. Gesture 7(1). 2007.
Magic Paper: Sketch-Understanding Research. Computer 40(9), pp.34--41. Los Alamitos, CA, USA, 2007.
Sketch Interpretation Using Multiscale Models of Temporal Patterns. IEEE Computer Graphics and Applications 27(1), pp.28-37. January 2007.
2005
LADDER, a sketching language for user interface developers. Elsevier, Computers and Graphics 28, pp.518-532. 2005.
Conference Papers2009
Symmetric Multimodal Interaction in a Dynamic Dialogue. In 2009 Intelligent User Interfaces Workshop on Sketch Recognition. February 2009.
Combining Speech and Sketch to Interpret Unconstrained Descriptions of Mechanical Devices. In Proceedings of the 2009 International Joint Conference on Artificial Intelligence (IJCAI), pp.1401-1406. Pasadena, California, July 2009.
Recognizing Interspersed Sketches Quickly. In Graphics Interface Conference 2009, pp.157-166. New York, NY, USA, May 25-27 2009.
Visual Recognition of Sketched Symbols. In 2009 Intelligent User Interfaces Workshop on Sketch Recognition. 2009.
2008
Discourse Topic and Gestural Form. In Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence, AAAI 2008, Chicago, Illinois, USA, July 13-17, 2008, pp.1-6. 2008.
Gestural Cohesion for Topic Segmentation. In Proceeings of ACL-08, pp.852-860. Columbus, Ohio, June 2008.
Modeling Gesture Salience as a Hidden Variable for Coreference Resolution and Keyframe Extraction. In Journal of Artificial Intelligence Research, pp.353-398. February 2008.
2007
Speech and Sketching: An Empirical Study of Multimodal Interaction. In SBIM '07: Proceedings of the 4th Eurographics workshop on Sketch-based interfaces and modeling, pp.83-90. New York, NY, USA, August 2-3 2007.
Conditional Modality Fusion for Coreference Resolution. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pp.352 - 359. 2007.
Turning Lectures into Comic Books with Linguistically Salient Gestures. In Proceedings of the Twenty-Second AAAI Conference on Artificial Intelligence, July 22-26, 2007, Vancouver, British Columbia, Canada. AAAI Press 2007, pp.877 - 882. 2007.
Recognition of Hand Drawn Chemical Diagrams. In Proceedings of the Twenty-Second AAAI Conference on Artificial Intelligence, July 22-26, 2007, Vancouver, British Columbia, Canada., pp.846-851. 2007.
2006
Gesture Features for Coreference Resolution. In Machine Learning for Multimodal Interaction: Third International Workshop, MLMI 2006, Bethesda, MD, USA, May 1-4, 2006, Revised Selected Papers (Lecture Notes in Computer Science), pp.154-165. Secaucus, NJ, USA, 2006.
Gesture Improves Coreference Resolution. In Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers, pp.37-40. New York City, USA, June 2006.
Interactive Learning of Structural Shape Descriptions from Automatically Generated Near-miss Examples. In Intelligent User Interfaces (IUI), pp.37-40. 2006.
Sketch Interpretation Using Multiscale Models of Temporal Patterns. In Proceedings of NESCAI 2006, pp.28-37. Ithaca, New York, April 28-29 2006.
2005
Dynamically Constructed Bayes Nets for Multi-Domain Sketch Understanding. In Proceedings of International Joint Conference on Artificial Intelligence (IJCAI)2005, pp.1407-1412. San Francisco, California, August 1 2005.
Gestural Cues for Sentence Segmentation. In AI Memo 2005-014, pp.1-14. Cambridge, MA, April 2005.
Gestural Features for Sentence Segmentation. In Proceedings of 6th European Gesture Workshop, pp.1-2. 2005.
HMM-Based Efficient Sketch Recognition. In Proceedings of the International Conference on Intelligent User Interfaces (IUI'05), pp.281-283. New York, New York, January 9-12 2005.
2004
Building the Design Studio of the Future. In AAAI 2004 Symposium on Making Pen-Based Interaction Intelligent and Natural, pp.1-7. Menlo Park, California, October 21-24 2004.
Speech and Sketching for Multimodal Design. In Proceedings of the 9th International Conference on Intelligent User Interfaces, pp.214-216. 2004.
Sketch Recognition User Interfaces: Guidelines for Design and Development. In AAAI 2004 Symposium on Making Pen-Based Interaction Intelligent and Natural, pp.8-14. Menlo Park, California, October 21-24 2004.
SketchREAD: A Multi-Domain Sketch Recognition Engine. In Proceedings of UIST 2004, pp.23-32. New York, New York, October 24-27 2004.
New Approach to Early Sketch Processing. In AAAI 2004 Symposium on Making Pen-Based Interaction Intelligent and Natural, pp.29-34. Menlo Park, California, October 21-24 2004.
A Salience-Based Approach to Gesture-Speech Alignment. In Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics, pp.25-32. East Stroudsburg, Pennsylvania, May 2-7 2004.
Visual and Linguistic Information in Gesture Classification. In International Conference on Multimodal Interfaces (ICMI'04), pp.113-120. New York, New York, October 14-15 2004.
Automatically Transforming Symbolic Shape Descriptions for Use in Sketch Recognition. In Proceedings of the Nineteenth National Conference on Artificial Intelligence (AAAI-04), pp.450-456. San Jose, California, 2004.
Shady: A Shape Description Debugger for Use in Sketch Recognition. In AAAI Fall Symposium on Making Pen-Based Interaction Intelligent and Natural, pp.71-77. Menlo Park, California, October 21-24 2004.
ETCHA Sketches: Lessons Learned from Collecting Sketch Data. In AAAI 2004 Symposium on Making Pen-Based Interaction Intelligent and Natural, pp.134-140. Menlo Park, California, October 21-24 2004.
Handling Overtraced Strokes in Hand-Drawn Sketches. In Making Pen-Based Interaction Intelligent and Natural, pp.141-144. Menlo Park, California, October 21-24 2004.
Scale-space Based Feature Point Detection for Digital Ink. In Making Pen-Based Interaction Intelligent and Natural, pp.145-151. Menlo Park, California, October 21-24 2004.
Perceptually Based Learning of Shape Descriptions. In Proceedings of the Nineteenth National Conference on Artificial Intelligence (AAAI-04), pp.482-487. San Jose, California, 2004.
2003
Natural Gesture in Descriptive Monologues. In Supplementary Proceedings of the ACM Symposium on User Interface Software and Techology (UIST'03), pp.69-70. New York, New York, November 2-5 2003.
LADDER: A Language to Describe Drawing, Display, and Editing in Sketch Recognition. In Proceedings of the 2003 International Joint Conference on Artificial Intelligence (IJCAI), pp.461-467. Acapulco, Mexico, 2003.
2002
A Framework for Multi-Domain Sketch Recognition. In AAAI Spring Symposium on Sketch Understanding, pp.1-8. Stanford, California, March 25-27 2002.
Sketch Understanding in Design: Overview of Work at the MIT AI Lab. In Sketch Understanding, Papers from the 2002 AAAI Spring Symposium, pp.24-31. Stanford, California, March 25-27 2002.
An Agent-Based System For Capturing and Indexing Software Design Meetings. In Proceedings of International Workshop on Agents In Design, WAID'02, pp.203-218. 2002.
Tahuti: A Geometrical Sketch Recognition System for UML Class Diagrams. In AAAI Spring Symposium on Sketch Understanding, pp.59-68. Stanford, California, March 25-27 2002.
2001
Preserving the Freedom of Sketching to Create a Natural Computer-Based Sketch Tool. In Human Computer Interaction International Proceedings. 2001.
Resolving Ambiguities to Create a Natural Computer-Based Sketching Environment. In Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence, pp.1365--1371. Seattle, Washington, USA, August 2001.
Query By Attention: Visually Searchable Information Maps. In Proceedings of Fifth International Conference on Information Visualization (InfoVis 2001), pp.85-93. 2001.
Naturally Conveyed Explanations of Device Behavior. In Workshop on Perceptive User Interfaces. 2001.
Sketch Based Interfaces: Early Processing for Sketch Understanding. In Workshop on Perceptive User Interfaces, Orlando FL, pp.1-8. 2001.
2009, pages = 1463-1468
A Visual Approach to Sketched Symbol Recognition. In Proceedings of the 2009 International Joint Conference on Artificial Intelligence (IJCAI). 2009, pages = 1463-1468.
Theses
MIDOS: Multimodal Interactive DialOgue System. Ph.D Thesis for Massachusetts Institute of Technology. June 2009.
Combining Representations for Improved Sketch Recognition. Ph.D Thesis for Massachusetts Institute of Technology. September 2009.
Drawing on the World: Sketch in Context. Master's Thesis for Massachusetts Institute of Technology. Cambridge, MA, September 2009.
Gesture in Automatic Discourse Processing. Ph.D Thesis for Massachusetts Institute of Technology. Camnridge, MA, May 2008.
LADDER: A Perceptually-based Language to Simplify Sketch Recognition User Interface Development. Ph.D Thesis for Massachusetts Institute of Technology. Cambridge, MA, January 2007.
Envisioning Sketch Recognition: A Local Feature Based Approach to Recognizing Informal Sketches. Ph.D Thesis for Massachusetts Institute of Technology. Cambridge, MA, May 2007.
Sketch Recognition for Course of Action Diagrams. Master's Thesis for Massachusetts Institute of Technology. Cambridge, MA, September 2007.
Using Context to Resolve Ambiguity in Sketch Understanding. Master's Thesis for Massachusetts Institute of Technology. Cambridge, MA, February 2006.
Sketch Interpretation Using Multiscale Stochastic Models of Temporal Patterns. Ph.D Thesis for Massachusetts Institute of Technology. May 2006.
Multi-Domain Sketch Understanding. Ph.D Thesis for Massachusetts Institute of Technology. August 2004.
Segmentation and Alignment of Speech and Sketching in a Design Environment. Master's Thesis for Massachusetts Institute of Technology. Cambridge, MA, February 2003.
Dr. Jones: A software Design Explorer's Crystal Ball. Ph.D Thesis for Massachusetts Institute of Technology. August 2003.
Perceptually Based Learning of Shape Descriptions. Master's Thesis for Massachusetts Institute of Technology. Cambridge, MA, 2003.
Feature Point Detection and Curve Approximation for Early Processing of Free-Hand Sketches. Master's Thesis. May 2001.
Department of EECS, MIT.
A Natural Sketching Environmant: Bringing the Computer into Early Stages of Mechanical Design. Master's Thesis for MIT. 2000.
Understanding Naturally Conveyed Explanations of Device Behavior. Master's Thesis for Massachusetts Institute of Technology. Cambridge, MA, 2000.
Designing Navigable Information Spaces. Master's Thesis for Massachusetts Institute of Technology. 1998.
MIT Reports and Publications2006
Initial Results from Speech and Sketching User Study. In Second Annual CSAIL Student Workshop. 2006.
2005
Modeling Sketching as a Dynamic Process. In CSW '05 Gloucester, MA. 2005.
2004
Using Speech and Sketching in a Design Environment. In MIT Computer Science and Artificial Intelligence Laboratory Annual Research Abstract. February 2004.
Automatically Generating Sketch Interfaces from Shape Descriptions. In Proceedings of the 4th Annual MIT Student Oxygen Workshop. 2004.
Won Best Paper.
Debugging Shape Definitions for Use in Sketch Recognition. In MIT Computer Science and Artificial Intelligence Laboratory Annual Research Abstract, pp.71-77. Cambridge, Massachusetts, September 2004.
LADDER: A Sketch Recognition Language. In MIT Computer Science and Artificial Intelligence Laboratory Annual Research Abstract. September 2004.
Testing Shape Descriptions by Automatically Translating them for Use in Sketch Recognition. In MIT Computer Science and Artificial Intelligence Laboratory Annual Research Abstract. September 2004.
Building a Training and Test Corpus of Hand Drawn Sketches. In MIT Computer Science and Artificial Intelligence Laboratory Annual Research Abstract. February 2004.
Efficient search space exploration for sketch recognition. In MIT Computer Science and Artificial Intelligence Laboratory Annual Research Abstract. 2004.
2003
Creating a Multimodal Design Environment Using Speech and Sketching. In Third Annual MIT CSAIL Student Oxygen Workshop. 2003.
Dynamically Constructed Bayesian Networks for Sketch Understanding. In Third Annual MIT CSAIL Student Oxygen Workshop. 2003.
Won Best Presentation.
Generic and HMM based approaches to freehand sketch recognition. In Proceedings of the MIT Student Oxygen Workshop. 2003.
Recognition efficiency issues for freehand sketches. Proceedings of the MIT Student Oxygen Workshop . 2003.
2002
Mutual Disambiguation of Verbal and Sketching Inputs in a Design Environment. In MIT Artificial Intelligence Laboratory Annual Abstract. September 2002.
Designs for the Future. In MIT Artificial Intelligence Laboratory Annual Abstract. September 2002.
Dr. Jones: A Software Design Explorer's Crystal Ball. In MIT Artificial Intelligence Laboratory Annual Abstract. September 2002.
A Domain Description Language for Sketch Recognition. In MIT Artificial Intelligence Laboratory Annual Abstract. September 2002.
A Domain Description Language for Sketch Recognition. Proceedings of the 2nd Annual MIT Student Oxygen Workshop . 2002.
Multi-Domain Sketch Recognition. Proceedings of the 2nd Annual MIT Student Oxygen Workshop . 2002.
Sketch Recognition in Software Design. In MIT Artificial Intelligence Laboratory Annual Abstract. September 2002.
Generating Domain Specific Sketch Recognizers From Object Descriptions. In MIT Artificial Intelligence Laboratory Annual Abstract. 2002.
Generating Domain Specific Sketch Recognizers From Object Descriptions. Proceedings of the MIT Student Oxygen Workshop . 2002.
2001
A Framework for Multi-Domain Sketch Recognition. In MIT Artificial Intelligence Laboratory Annual Abstract. September 2001.
Graph Exploration for Software Archeology. In MIT Artificial Intelligence Laboratory Annual Abstract. September 2001.
Ligature: Gesture-Based Configuration of the E21 Intelligent Environment. In MIT Artificial Intelligence Laboratory Annual Abstract. September 2001.
Ligature, Gesture-Based Configuration of the E21 Intelligent Environment. Proceedings of the MIT Student Oxygen Workshop . 2001.
Query by Attention: Visually Searchable Information Maps. In MIT Artificial Intelligence Laboratory Annual Abstract. September 2001.
Natural Editing and Recognition of UML Class Diagrams. In MIT Artificial Intelligence Laboratory Annual Abstract. September 2001.
Natural Sketch Recognition in UML Class Diagrams. Proceedings of the MIT Student Oxygen Workshop . 2001.
Free-Hand Stroke Approximation for Intelligent Sketching Systems. Proceedings of the MIT Student Oxygen Workshop . 2001.
Scale-space Based Feature Point Detection for Noisy Digital Curves. In MIT Artificial Intelligence Laboratory Annual Abstract. 2001.
2000
Intelligent Mechanical Engineering Design Environment: From sketching to simulation. In MIT Artificial Intelligence Laboratory Annual Abstract. September 2000.
Supporting Group Brainstorming. In MIT Artificial Intelligence Laboratory Annual Abstract. September 2000.
Understanding Naturally Conveyed Explanations of Device Behavior. In MIT Artificial Intelligence Laboratory Annual Abstract. September 2000.
Early Processing in Sketch Understanding. In MIT Artificial Intelligence Laboratory Annual Abstract. 2000.
1999
Behavior Annotated Designs: A multimodal approach to design rational capture and intelligent design environments. In MIT Artificial Intelligence Laboratory Annual Abstract. September 1999.
Other Work
A Domain Description Language for Sketch Recognition.March, 19 2003.
PhD Thesis Proposal, Massachusetts Institute of Technology, to be published.
Dr. Jones: A Software Archaeologist's Magic Lens.May 2001.
PhD Thesis Proposal, Department of EECS, MIT. Movies
Early demonstration of the ASSIST sketch understanding system. [mpg format; 49 Meg]
This is done using a Mimio/whiteboard; the current system enables a more natural sketching style and operates on either a whiteboard and a tablet computer.
|
