Proceedings of the Annual Meeting of the Cognitive Science Society

Knowledge about the functional properties of the world constraints and informs perception. For example, looking at a table, chair, a building or a sculpture, we are able to resolve occluded attachments because we know that in order to stand, an object's center of gravity must lie within its footprint. W h e n when we see a floating wheel in the interior of a vehicle, we know that it is probably the mejuis by which the driver communicates steering information to the chassis. Movable handles imply input to machines; fixed handles imply an upside and a downside to any object they grace. We are constructing a machine-understanding machine with which to explore the usefulness of semantics in perception. This system will investigate simple mechanical devices such as geai trains, simultaneously building a representation of the structures and functions of parts, and using that representation to guide and disambiguate perception. In this paper we discuss how this work has led to an understanding of perception in which a semantics of structure and function play a central role in guiding even the lowest level perceptual actions.