A Sentence is Worth a 3-D Picture

NSF-funded computer scientists and education researchers have developed a system that generates 3-D scenes from written sentences. They have shown the system is a useful tool to improve middle school students' understanding of literature.

Use of the system will contribute to the study of how individuals learn language as well as the development and evaluation of educational software. The underlying advances are relevant to a broad range of technologies requiring accurate interpretation of language in context such as robotics; computing and communications devices; and games.

The system, called WordsEye, analyzes input (written sentences) and then associates the input with 3-D objects from a library. Analysis includes assigning meaning to different verbs depending on information derived from a rich database. Details for the database come from existing databases of nouns, verbs, adjectives and adverbs as well as automated searches of internet sources such as Wikipedia.

Sixth-graders tested WordsEye as part of a college-preparatory enrichment program run by the Harlem Educational Activities Fund. The students used the system for 90 minutes a week over 5 weeks to create scenes from the literature they read, including Aesop's Fables and Animal Farm. Compared to other students enrolled in the same course, the students using WordsEye wrote better final essays based on the literature they read.