Intelligent software that brings rough sketches to life in a virtual world
is promising to revolutionise the way children learn and to help engineers
visualise their designs.
In designing the software, developers at the Massachusetts Institute of
Technology have had to tackle several tough tasks. First the software must
recognise crude hand-drawn shapes in the way the user intended for example,
by spotting that four wiggly lines represent a square with straight sides.
Then it has to recognise the context of the objects being sketched, in
the same way that we interpret two circles under a box on a slope as some
kind of vehicle on wheels. Finally, the package has to animate the sketch
so that objects move as they would do in the real world.
Current sketching software only recognises the geometry of a sketch -
for example, that two lines meet at a specific angle - and cannot recognise
context at all.
Allocation and adjustment
The MIT software monitors the image as it is being drawn on to a computer
screen and allocates probabilities to various interpretations of what it
As the user adds more detail, the software adjusts these weightings. To
do this, it uses a technique known as Bayesian analysis, which is normally
used to compute the likelihood of specific causes, given certain effects.
Subscribe to New Scientist for more news and features
"With our software, the 'causes' are what the user had in mind to draw,
and the 'effects' are what was actually drawn," says Randall Davis, who has
developed the code with Christine Alvarado at MIT's Artificial Intelligence
Once the computer has settled on an interpretation, it applies the laws
of physics such as gravity and friction to set the virtual world in motion.
Cars roll down hills, pendulums swing, and objects bang into each other.
The Bayesian part of the software is still at an early stage: it recognises
only squares. Davis's next goal is to get it to recognise a much broader
range of crudely drawn shapes.
Part of the problem is thinking up all the possible ways someone might
draw a shape, so as to be able to assign realistic probabilities to particular
patterns on the screen.