Smart software makes sense of rough sketches

Intelligent software that brings rough sketches to life in a virtual world is promising to revolutionise the way children learn and to help engineers visualise their designs.

In designing the software, developers at the Massachusetts Institute of Technology have had to tackle several tough tasks. First the software must recognise crude hand-drawn shapes in the way the user intended for example, by spotting that four wiggly lines represent a square with straight sides.

Then it has to recognise the context of the objects being sketched, in the same way that we interpret two circles under a box on a slope as some kind of vehicle on wheels. Finally, the package has to animate the sketch so that objects move as they would do in the real world.

	Animating sketches

Current sketching software only recognises the geometry of a sketch - for example, that two lines meet at a specific angle - and cannot recognise context at all.

Allocation and adjustment

The MIT software monitors the image as it is being drawn on to a computer screen and allocates probabilities to various interpretations of what it might represent.

As the user adds more detail, the software adjusts these weightings. To do this, it uses a technique known as Bayesian analysis, which is normally used to compute the likelihood of specific causes, given certain effects.

	Subscribe to New Scientist for more news and features
	Related Stories   Wheelchair moves at the speed of thought 24 July 2003   Software can investigate suspicious deaths 7 July 2003   Intel to release machine learning libraries 2 May 2003     For more related stories search the print edition Archive     Weblinks   Artificial Intelligence Laboratory, MIT   Bayesian logic

"With our software, the 'causes' are what the user had in mind to draw, and the 'effects' are what was actually drawn," says Randall Davis, who has developed the code with Christine Alvarado at MIT's Artificial Intelligence Laboratory.

Once the computer has settled on an interpretation, it applies the laws of physics such as gravity and friction to set the virtual world in motion. Cars roll down hills, pendulums swing, and objects bang into each other.

The Bayesian part of the software is still at an early stage: it recognises only squares. Davis's next goal is to get it to recognise a much broader range of crudely drawn shapes.

Part of the problem is thinking up all the possible ways someone might draw a shape, so as to be able to assign realistic probabilities to particular patterns on the screen.

Celeste Biever

For more exclusive news and expert analysis every week subscribe to New Scientist print edition.