We have built a system that acquires semantics for spatial terms in a simple 2D world. Users can select objects and describe them relative to a landmark (here marked by a pear). For example, someone might pick the apple circled here and say “a little above”. The system uses spoken input processed through our speech recognition system.
Ôøº
On the visual side, we measure a set of language independent features that includes not only the obvious distance and angle measurements, but additional values indicating, for example, the shape of the landmark.Ôøº
