How Space Structures Language[1]
Stanford University Department of Psychology, Bldg. 420
Stanford, California
94305-2130
Abstract. As Talmy has observed, language schematizes space; language provides a systematic framework to describe space, by selecting certain aspects of a referent scene while neglecting the others. Here, we consider the ways that space and the things in it are schematized in perception and cognition, as well as in language. We propose the Schematization Similarity Conjecture: to the extent that space is schematized similarly in language and cognition, language will be successful in conveying space. We look at the evidence in both language and perception literature to support this view. Finally, we analyze schematizations of routes conveyed in sketch maps or directions, finding parallels in the kind of information omitted and retained in both.
1 Introduction
Language can be effective in
conveying useful information about unknown things. If you are like many people, when you go to a new place, you
may approach a stranger to ask directions. If your addressee in fact knows how to get to where you want
to go, you are likely to receive coherent and accurate directions (cf. Denis,
1994; Taylor and Tversky, 1992a).
Similarly, as any Hemingway reader knows, language can be effective at
relating a simple scene of people, objects, and landmarks. In laboratory settings,
narratives relating scenes like these are readily comprehended. In addition, the mental representations
of such scenes are updated as new descriptive information is given (e. g.,
Glenberg, Meyer, and Lindem, 1987;
Morrow, Bower and Greenspan, 1989). Finally, times to retrieve spatial information from mental
representations induced by descriptions are in many cases indistinguishable
from those established from actual experience (cf. Franklin and Tversky, 1990;
Bryant, Tversky, and Lanca, 1998).
Contrast these successful uses of language with another one. You've just returned from a large party
of both acquaintances and strangers.
You try to describe someone interesting whom you met to a friend because
you believe the friend knows this person's name. Such descriptions are notoriously poor. In fact, in some situations, describing
a face is the surest way to reduce memory for it (Schooler and Engstler-Schooler,
1991). Why is it that language is
effective for conveying some sorts of spatial information but not others?
The answer may lie in the way that language structures
space. In 1983, Leonard Talmy published an article with that title which has
rippled through cognitive psychology and linguistics like a stone skipped on
water. In it, he proposed that language
"schematizes" space, selecting "certain aspects of a referent
scene...while disregarding the remaining aspects."(p. 225). For example, a term like
"across" can apply to a set of spatial configurations that do not
depend on exact metric properties such as shape, size, and distance. Use of "across" depends on
the global properties and configuration of the thing doing the crossing and the
thing crossed. Ideally, the thing
doing the crossing is smaller than the thing being crossed, and it is crossing
in a straight path perpendicular to the length of the thing being crossed. Thus schematization entails information
reduction, encoding certain features of a scene while ignoring others. Talmy's analysis of schematization
focused on the fine structure of language, in particular, closed-class terms,
and less on the macroscopic level of sentences, paragraphs and discourse that
uses a language's large set of open-class lexical items as elements. Closed-class grammatical forms include
"grammatical elements and categories, closed-class particles and words,
and the syntactic structures of phrases and clauses." (p. 227). Despite their syntactic status, they
express meanings, but only limited ones, including space, time, and
perspective, important to the current issues, and also attention, force,
causation, knowledge state, and reality status. Because they appear across languages, they are assumed to
reflect linguistic, hence cognitive, universals.
Not only language, but also perception and conception, which
Talmy has collectively called Ôception,
schematize space and the things in it (Talmy, 1996). In the following pages, we first examine how 'ception
schematizes. Then, we go on to examine how the schematization of 'ception maps
onto language. There is no
disputing that language is a powerful clue to 'ception, that many of the
distinctions important in 'ception are made in language, some in closed-class
terms, others in lexical items.
Yet, there are notable exceptions.
As observed earlier, people are poor at describing faces, though
excellent at recognizing them, a skill essential for social interaction. In contrast, routes and scenes are more
readily conveyed by language despite the fact that, like faces, routes and
scenes consist of elements and the spatial relations among them. Here, we propose a conjecture, the
Schematization Similarity Conjecture:
To the extent that language and Ôception schematize things similarly,
language will be successful at communicating space.
To understand how 'ception schematizes space is to
understand that perception is not just bottom-up, determined by the stimulus input
alone, but is in addition top-down, conditioned by what is already in the mind,
momentarily and longterm. Therefore,
any generalizations based on schematizations of space necessarily lead to oversimplifications.
One of these is ignoring context.
It has long been clear, but is sometimes overlooked, that how people
perceive of, conceive of, and describe a scene is deeply affected by a wealth
of nonindependent factors, including what they are thinking, how they construe
the scene, the goals at hand, past experience, and available knowledge
structures.
Despite the fact that language and 'ception always occur in
a context, there seem to be levels of schematization that hold over many
contexts. People do not reinvent vocabulary and syntax at every encounter. If they did, communication would not be
possible. Schematization in
language and in 'ception is always a compromise; it must be stable enough for
the general and the venerable, yet flexible enough for the specific and the
new. In the following sections, we will review the existing research on how
both 'ception and language schematize space and objects in it, abstracting
certain features and ignoring others.
This review of schematization will be schematic itself. It will be an attempt to give the
"bottom line," the general aspects of objects and space most critical
to our understanding of them. The
evidence comes from many studies using different techniques and measures, that
is, different contexts. Some of
this evidence rests on language in one way or another. Ideally, evidence based purely on
perception could be separated from evidence resting on language in order to
separate the schematization of perception alone from that influenced by
language. But this is probably not
possible. For one thing, using
non-linguistic measures is no guarantee that language is not implicitly
invoked. With these caveats in
mind, let us proceed to characterize how the things in the world and the
spatial relations among them are schematized.
2 Figures, Objects, Faces
When we look at the world around us,
we don't see it as a pattern of hues and brightnesses. Rather, we perceive distinct figures
and objects. For human perceivers,
then, space is decomposed into figures and the spatial relations among them,
viewed from a particular perspective.
Similarly, figures can be decomposed into their parts and the spatial
relations among them. Our
experience of space, then, is not abstract, of empty space, but rather of the
identity and the relative locations of the things in space.
2.1 Figures
There are two major questions in
recognition of the things in space.
First, how do we get from retinal stimulation to discernment of figures?
This is the concern of the Figures
section. Next, how do we get from
a view-dependent representation to a view-independent representation? This is
the concern of the Objects
section. One of the earliest
perceptual processes is discerning figures from background (e. g., Hochberg,
1978; Rock, 1983). Once figures
are identified, they appear closer and brighter than their backgrounds. In contrast to grounds, figures tend to
have closed contours and symmetry, so the Gestalt principles of figurality,
including continuity, common fate, good form, and proximity, all serve as
useful cues. Thus, the eye and the
brain look for contours and cues to figurality in pursuit of isolating figures
from grounds. Another way to put
this is that figures are schematized as contours that are likely to closed and
likely to be symmetric.
Language for Figures.
The distinctions that Talmy elucidates
begin with figure and ground. Talmy borrows these terms from their use in
perception and Gestalt psychology described above. Just as perception focuses on figures, so does language,
according to Talmy. He argues that
language selects one portion of a scene, the figure, as focal or primary,
and describes it in relation to
another portion, the ground, and sometimes in addition in relation to a third
portion of the scene. We say, for example, "the horse is by the barn"
or "the horse is near the trough in front of the barn." The figure is conceived of as
geometrically simpler than the ground, often only as a point. It is also usually smaller, more
salient, more movable, and more recent than the ground, which is more permanent
and earlier. Although the ground
is conceived of as geometrically more complex than the figure, the ground, too,
is schematized, as indicated in English by prepositions, a closed-class
form. For example, "at"
schematizes the ground to a point, "on" and "across" to a two-dimensional surface,
"into" and "through" to a three-dimensional volume.
A comparison between 'ception and language of figures shows
a number of similarities and differences.
Both divide the world into figures and ground, introducing asymmetries
not present in the world per se.
In 'ception, figures appear closer and brighter than grounds, becoming
more salient. In language, figures
are the primary objects currently salient in attention and discourse. Nevertheless, the object that is figural
in perception may not be figural in language. An example comes from unpublished eye movement data
collected by Griffin (Z. Griffin, 1998, personal communication). In scanning a picture of a truck about
to hit a nurse, viewers fixate more on the truck, as the agent of the
action. Yet, the nurse is the
figure in viewers' descriptions of the scene. In addition, figures in 'ception are conceived of as shapes
with closed contours and often symmetric, yet in language, they are often
reduced to a point in space.
2.2 Objects
The human mind does not seem content
with simply distinguishing figures from grounds; it also identifies figures as
particular objects. But objects
have many identities. What we typically
sit on can be referred to as a desk chair, or a chair, or a piece of
furniture. Despite the
possibilities, people are biased to identify objects at what has been called
the ÒbasicÓ level (e.g., Brown, 1958; Murphy and Smith, 1982; Rosch,
1978). This is the level of chair,
screwdriver, apple, and sock rather than the level of furniture, tool, fruit,
and clothing, or the level of easy chair, Phillips-head screwdriver, delicious
apple, and anklet. This is the
level at which people seem to have the most information, indexed by attribute
lists, relative to the number of alternative categories that must be kept in
mind.
Many other cognitive operations also converge at the basic
level. It is the level at which
people are fastest to categorize instances (Rosch, 1975), the level fastest to
identify (Murphy and Smith, 1982), the level people spontaneously choose to
name, the highest level of abstraction for which an outline of overlapped
shapes can be recognized, the highest level for which there is a common set of
behaviors, and more (Rosch, 1978; Rosch, Mervis, Gray, Johnson, and
Boyes-Braem, 1976). The basic
level, then, has a special status in perception, in behavior, and in language
(Tversky, 1985; Tversky and Hemenway, 1984). Rosch (1978) suggested that the natural breaks in labeling
are based in the natural breaks in objects as we perceive them given our
perceptual apparatus. Features of
objects are not uniformly distributed across classes of objects. Instead,
features of objects are correlated, that is, things that have feathers and
beaks also lay eggs and fly.
The natural level for identifying objects, then, is the
basic level. Arriving at view-independent representations of objects requires
more than the visual input alone; it also requires some more general knowledge
about the objects in question (e. g., Marr, 1982). As for figures, contour and symmetry characterize particular
objects, but with greater specificity. Basic objects, such as couches and
socks, can be recognized from a set of overlapping instances, standardized for
size and viewpoint (Rosch, et al., 1976).
Shapes of different kinds of socks are quite similar, but quite
different from shapes of other objects even from the same category, such as shirts
or ties. Furthermore, objects are
most easily recognized when they are viewed from a canonical orientation,
upright, and typically 3/4 view (Palmer, Rosch, and Chase, 1981). This view is one that presents the
greatest number of features characteristic of the object. In many cases, those characteristic
features are parts of the object (Biederman, 1987; Tversky and Hemenway, 1984);
the greater the number of object parts detectable, the easier the identification
of the object (Biederman, 1987).
Parts have a dual status in cognition. On the one hand, they are perceptually salient as they are
rooted in discontinuities of object shape (e. g., Biederman, 1987; Hoffman and
Richards, 1984). On the other
hand, different parts have different functions and serve different purposes to
humans (Tversky and Hemenway, 1984).
Parts are at once components of perception and components of function
and facilitate inferences from appearance to behavior. Symmetry, too, is used to identify
specific objects. Viewers
interpret asymmetric nonsense figures as upright, off-center views of symmetric
objects (McBeath, Schiano, and Tversky, 1997). 'Ception, then, schematizes specific figures, that is,
objects, as shapes, composed of parts, and most likely upright and symmetric.
Language for Objects.
Objects are typically named by open-class terms, thus not considered by
Talmy. Perhaps individual objects
are not an inherent part of the structure of language because there are so many
of them and many of those are context specific. The place-holder for individual objects, nouns or subjects,
is, of course, part of language structure as are various operations on them,
such as pluralizing. Nevertheless,
there are clues to way objects are conceived in the ways that names for objects
are extended. Shape seems to be a
primary basis for categorization as well as for extension of object terms, in
both children's "errors" and adults' neologisms (Clark, 1973; Clark
and Clark, 1979; Bowerman, 1978a, 1978b).
There are old examples, like "stars" and "hearts"
that are not really shaped like stars or hearts. And there are new examples, such as the body types loved by
cardiologists--"pear-shaped"--and that disparaged by
cardiologists--"apple-shaped,"--affectionately called simply
"pears" and "apples."
2.3 Faces
Faces are a special kind of object
in several ways. Recognition of
faces is most typically at the level of the individual, not at the level of the
class. For example, when we talk
about identifying or recognizing a face, we mean recognizing that a specific
face is the current president of the United States and not his brother. In contrast, when we talk about
recognizing an object as a chair, we're usually not concerned with whose chair
or even what type of chair. Of
course, we need to identify some objects other than faces at the level of the
individual. But identifying my
house or car or jacket is facilitated by features such as locations or color or
size, and such features may not facilitate identifying specific faces. Faces, in addition, are not integral
objects in and of themselves, they are parts of other objects, human or
otherwise. Recognizing faces is
dependent on internal features, not just an outline shape. This is why we see faces not only in
the moon, which has the proper outline, but also in cars, which do not. Furthermore, the features need to be in
the proper configuration. Changing
the overall configuration leads to something that is not a face, and even
altering the relative distances among properly configured features diminishes
resemblance substantially (cf. Bruce, 1988). For identifying individuals, in addition to configuration of
features, the shapes of component features are also important, and those shapes
are not regular. Similar to
objects, 3/4 views are best recognized in faces (e. g., Hagen & Perkins,
1983; Shapiro & Penrod, 1986), perhaps because a 3/4 view gives better
information about important component features, such as shape of nose, chin,
and forehead. Even more than for
objects, orientation is important in faces; upside down faces are considerably
harder to recognize than right side up (e. g., Carey and Diamond, 1987; Yin,
1969). Turning objects upside down
seems to be more disruptive to objects with irregular internal features such as
faces than to objects with horizontal and vertical internal features like houses. Schematization of individual faces,
then, is far more precise, entailing orientation as well as configuration and
shapes of internal features.
Language for Faces. As
noted earlier, faces are often perceived at the level of the individual. Similarly, they are referred to by open
class terms, that is, names of individuals. In contrast to names for objects, when names of individuals
are extended, it is typically personality traits or personal history that is
extended, not shape as for objects, in fact, not appearance at all (cf. Clark
and Clark, 1979). Identifying
faces requires 'ception of subtle spatial relations among the parts (e. g.,
Bruce, 1988). Language, however,
schematizes spatial relations in cruder categories, such as above, below,
front, back, near, far, between, and among. Finer distinctions can be made but in the technical language
of measurement. In addition,
estimates of fine measurement are frequently unreliable (e. g., Leibowitz,
Guzy, Peterson, and Blake, 1993). Thus, the puzzle that language is adequate
for conveying routes but inadequate for describing faces is solved. The spatial relations usually needed
for getting around are readily captured by language but the subtle spatial
relations needed for identifying faces are not readily schematized by language.
2.4 Summary
of Figures, Objects, and Faces.
For detecting figures, contour
(especially closed contour) and symmetry are among the diagnostic
features. For objects which are
figures identified at the basic level, specific contours or shapes that are
decomposable into parts are characteristic, along with orientation and
symmetry. For faces which are
parts of objects identified at the level of an individual, the internal
configuration and shapes of features is critical, in addition to orientation
and symmetry. Returning to language, note that
figures and objects are named by open-class terms, as are grounds. These refer to classes of things, and,
interestingly, are sometimes extended to refer to shapes (as in
"pear-shaped"). Faces,
by contrast, are called by names that refer to individuals, not classes, much
like street addresses, and that have no perceptual interpretation other than
reference to the individual. Names
for objects and faces, though less schematized than closed-class terms, are
nevertheless schematized. A table
is a table regardless of point of view, of color, of material, of location, to
a large extent of size. This is
not to say that people cannot or do not remember individual objects with their
specific features and locations, but that people generally think about and
refer to objects more abstractly.
3 Spatial Relations
Thus far, we have discussed the
elements in space and their schematization in 'ception and language. Knowledge and schematization of space
also entail the spatial relations among elements. In fact, we observed that entities can be decomposed into
parts and the spatial relations among them, and that as entities are identified
at more specific levels, the spatial relations among the parts become more
critical. In this section we turn
to the schematization of spatial relations. In perceiving a scene, figures are not just discerned and
identified, they are also located.
Figures are not located in an absolute way, but rather relative to other
reference figures and/or a frame of reference. We note, for example, that we left the car by a particular
street sign or that we buried the family heirlooms in the middles of a circle
of trees. Locating figures
relatively makes sense if only because perception of a scene is necessarily
dependent on a particular viewpoint, yet a view-independent representation of a
scene is desirable in order to recognize a scene or object from other
viewpoints. Reference objects and reference frames serve to schematize the
locations of figures. Memory for
orientations and locations of dots, lines, or figures is biased toward
reference objects or frames (e. g., Howard, 1982; Huttenlocher, Hedges, and
Duncan, 1991; Nelson and Chaiklin, 1980; Taylor, 1961; Tversky, 1981).
How are reference objects and frames selected? Proximity, salience, and permanence are
influential factors (Tversky, 1981; Tversky, Taylor, and Mainwaring,
1997). Domain, semantic, and
pragmatic factors, such as current goals and recent experience, can also affect
choice. Reference objects are
other figures in the same scene as the target object whereas reference frames
tend to surround the scene, the set of figures, in some way.
Natural borders and axes often serve as reference frames,
such as the sides of a room, the sides of a piece of paper, the land and the
sky. Horizontal and vertical lines or planes are privileged as reference
frames, whether actual or virtual, as in the sides of a page or map at odd
orientations. Acuity is better for horizontal and vertical lines, as is memory,
and both perception and memory are distorted toward them (see Howard, 1982 and
Tversky, 1992 for reviews).
Horizontal and vertical lines are relatively easy for children to copy,
but diagonal lines cause difficulties and are drawn toward horizontal and
vertical (Ibbotson and Bryant, 1976).
The human body, especially one's own, also serves as a natural reference
object. The projections of the
natural horizontal and vertical axes of the body, head/feet, front/back, and
left/right, are a privileged reference frame, with certain of the axes more
accessible than others, depending on body posture and viewpoint (e. g., Bryant,
Franklin and Tversky, 1992; Bryant, Tversky and Lanca, 1998; Franklin and
Tversky, 1990). Regions defined by
the axes also vary depending on viewpoint; for example, for self, front is
larger than back, and both are larger than left, and right, but not for
other.(Franklin, Henkel and Zangas, 1995).
3.1 Schematization and Language of Spatial Relations.
Spatial relations, then, are
schematized toward reference objects and frames, especially horizontal and
vertical planes. Spatial relations
are frequently but not always referred to by closed-class terms, prepositions,
such as "at," "on," and "in," or "in front
of," "on top of," "across," "near,"
"between," and "parallel to." The schematization of closed-class elements is topological,
according to Talmy. It abstracts
away the metric properties of shape, size, angle, and distance, distinctions
that are normally expressed in lexical elements. Talmy's analyses have been extended by others, especially in
the direction of examining the topological constraints underlying prepositions,
that is, the expression of spatial relations between a figure and a ground (e.
g., Herskovits, 1986, Lakoff, 1986; Landau and Jackendoff, 1993; Vandeloise,
1986). Some languages, however,
don't have prepositions. Even in
English, which does, open class terms also describe spatial relations, as in
"support," "hold," "lean," or
"approach."
The scene alone does not determine how it is schematized to
spatial relations, though it is often presupposed that the perceptual array is
primary (e. g., Carlson-Radvansky and Irwin, 1993; Logan and Sadler, 1996; Hayward and Tarr, 1995). The speaker's perspective, intent, and goals, as well as
cultural practices, are some of the influences on schematization. The interpretation of the scene in
light of current goals and cultural practices are among the influences on
selection of spatial relation terms.
As Talmy (1983) noted, we can go "through" or
"across" a park, and get "in" or "into" a car.
Appropriateness of words like "near" or "approach" depend
on the nature of the figure and the ground (Morrow and Clark, 1988). What's more, abstract uses of prepositions depend entirely
on functional, not spatial relations, as in "on welfare" or "in
a bad mood" (Garrod and Simon, 1989). Even spatial uses have a functional basis. One can say "the pear is in the
bowl" where the expression is even though in fact the pear is outside the
bowl on top of a pile of fruit.
This is because the pear's location is controlled by the location of the
bowl (Garrod and Simon, 1989).
Although the qualities of schematization of spatial relations in both language and perception are
similar, the open-class terms that are used to refer to figures preserve far
more detailed spatial information than the terms used to refer to spatial
relations. Moreover, although
memory for spatial location and orientation is biased toward reference frames and
objects, it does not coincide with them.
The schematization of the language of spatial relations may be in the
same directions as the schematization of 'ception of spatial relations, but it
is far more extreme.
4 Motion
Figures in space are not necessarily
static, nor are viewers. Perceiving and conceiving of motion are needed from
the beginning of life, and, in fact, motion in concert is another clue to
figurality (e. g., Spelke, Breinlinger, Macomber, and Jacobson, 1992). Perceiving motion accurately is not a simple matter. For example, generations of paintings
of horses galloping have portrayed their legs in impossible
configurations. When motion is
relatively simple, as in the path of a pendulum or a falling object, people are
able to recognize correct and incorrect paths of motion. Yet, some people correctly recognizing
paths of motion may nevertheless produce incorrect paths, indicating flawed
conceptions of motion (Kaiser, Proffitt, Whelan, and Hecht, 1992). Although
motion is continuous, people seem to conceive of it as sequences of natural
chunks (Hegarty, 1992). And
although motion is continuous, people tend to conceive of it hierarchically (e.
g., Newtson, Hairfield, Bloomingdale, and Cutino, 1987; Zacks and Tversky,
1997). As for objects, there seems
to be a preferred or basic level, the level of going to a movie (Morris and
Murphy, 1990; Rifkin, 1985). Although more can be said about actions and events,
we focus here on schematization of motion in 'ception and language.
4.1 Schematization
of Motion.
Many aspects of motion, such as
frequency and causality, are carried by closed-class terms (Talmy, 1975, 1983,
1985, 1988), yet other aspects of motion are referred to by open-class terms,
particularly verbs. Verbs vary
notoriously within and across languages as to what features they code (e. g.,
Gentner, 1981; Huttenlocher and Lui, 1981; Talmy, 1975, 1985, 1988). For example, some languages like
English regularly encode manner of motion in verbs, as in "swagger,"
"slink," "slide," and "sway," others primarily
encode path in verbs, as in "enter," "exit," and
"ascend" (Talmy, 1985).
Choice of verb is open to construal. The same perceptual sequence, such as leaving a room may be
described in many different ways (Gentner, 1981), such as "went,"
"raced," "stumbled," "cried," "got
chased," "got pushed," or "escaped" out the door. Although activities, like objects, are
conceived of hierarchically, descriptors of activities are not necessarily
organized hierarchically.
Huttenlocher and Lui (1981) have argued that verbs, in contrast to the
nouns used to refer to objects, are organized more as matrices than as
hierarchies.
Like figures, motion can be schematized at various levels of
specificity. The simplest way of
thinking about motion is the path of an entire figure, a point moving in
space. Like objects, paths are
perceived in terms of frames of reference and distorted toward them. Just as in locating objects, in
perceiving paths of motion, horizontal and vertical coordinates often serve as
a reference frame (e. g., Pani, William, and Shippey, 1995; Shiffrar and
Shepard, 1991). A more complex
level of schematization than a path of motion is a pattern of parts moving in
relation to one another. This
level is analogous to schematizing an object as a configuration of parts. It is the level of understanding of
pulleys (Hegarty, 1992) or gears (Schwartz and Black, 1996) or of
distinguishing walking from running, which people readily do from patch-light
displays (e. g., Cutting, Proffitt, and Kozlowski, 1978; Johansson, 1975). Yet another level of
schematization is manner of motion, as in distinguishing modes of walking, such
as swaggering or slinking.
5 Route
Directions and Maps
The simplest schematization of
motion to a path or route is readily encoded in language (e. g., Denis, 1994;
Levelt, 1982; Linde and Labov, 1975; Klein, 1982; Perrig and Kintsch, 1985;
Talmy, 1975; Taylor and Tversky, 1992a, 1992b, 1996; Wunderlich and Reinhelt,
1982). Routes are schematized as a
point changing direction along a line or a plane, or as a network of nodes and
links. Though by no means identical with perceptual or conceptual
schematization, route maps can be regarded as schematizations that are closer
to externalizations of perceptions than descriptions. Depictions of routes use spatial relations on paper to
represent spatial relations in the world.
Moreover, they can use iconic representations of entities in the world
to represent those entities. Routes, then, can be externally represented as
descriptions or depictions. Like
route directions, route maps are commonly used to convey how to get from A to
B. Which is better seems to depend
on the specifics of the navigation task (e. g., Streeter, Vitello, and
Wonsiewicz, 1985; Taylor, Naylor, and Chechile, in press; Taylor and Tversky,
1992a). Both route directions and route maps, then, seem adequate to convey
information sufficient for arriving at a destination. We were interested in whether descriptions and depictions of
routes schematize them similarly.
To get at this question, we approached students outside a
campus residence and asked them if they knew how to get to a popular off-campus
fast-food restaurant. If they did,
we handed them a piece of paper, and asked them to either write down the
directions or sketch a map. We
obtained a total of 29 maps and 21 directions. Sample descriptions appear in Table 1 and sample maps in
Figures 1. Note that route maps differ from other kinds of sketch maps in that
they contain only the paths and landmarks relevant to the specific route.
Following Denis (1994), we broke down the depictions and descriptions into
segments consisting of four elements each: start point, reorientation (direction), path/progression,
and end point. As the paths are
continuous, the start point for one segment served as the start point for the
next. In this situation, the
segments corresponded to changes of direction (action) in the route. It would be possible to have segments
separated by, say, major intersections or landmarks without changes in
direction, but this did not happen in this corpus. Because the sketch maps, unlike street maps, contained very
little information about the environment not directly related to the path, it
was not difficult to segment the maps. As defined, each segment contains
sufficient information to go from node to node. Together, these segments contain the information essential
to reach the destination. Two coders coded the maps and descriptions for these
categories of information and for categories of supplementary information. They first coded a subset of the
protocols, and after reaching agreement on those, coded the rest separately.
Table
1. Examples of Route Directions
|
DW 9 From Roble parking lot R onto Santa Theresa L onto Lagunita (the first stop sign) L onto Mayfield L onto Campus drive
East R onto Bowdoin L onto Stanford Ave. R onto El Camino go down few
miles. it's on the right. BD 10 Go down street toward
main campus (where most of the buildings are as opposed to where the
fields are) make a right on the first real street (not an entrance to a
dorm or anything else). Then make a left on the 2nd street you come
to. There should be some
buildings on your right (Flo Mo) and a parking
lot on your left. The street
will make a sharp right. Stay on it. that puts you on Mayfield road. The first intersection after the
turn will be at Campus drive.
Turn left and stay on campus drive until
you come to Galvez Street. Turn Right. go down until you get to El
Camino. Turn right (south) and
Taco Bell is a few miles down on the
right. BD 3 Go out St. Theresa turn Rt. Follow Campus Dr. way
around to Galvez turn left on Galvez. turn right on El
camino. Go till you see Taco
Bell on your Right |
Figure 1.
Examples of Route Maps
5.1 Essential Information
in Descriptions and Depictions.
Not all of the information included
in both maps and directions fit into the essential four categories. In fact, 91% of the people giving
directions and 90% of the people sketching maps added some information in addition
to the start and end points, reorientation and path/progression. The additional information for maps
included cardinal directions, arrows, distances, and extra landmarks. That same kind of information was added
to directions. In addition, some
directions also contained detail describing the landmarks and paths. This information, while not essential,
may be important for keeping the traveler confidently on track. It anticipates that travelers may
become uneasy when there is a relatively long distance without a change of
orientation or distinguishing feature or when there is uncertainty about the
identity of a landmark. The
descriptions obtained by Denis (1994) and by Gryl (1995) had the same character.
5.2 Schematizing
in Descriptions and Depictions.
Not only did the same critical and
supplementary information constitute the majority of content in route
descriptions and depictions but also that information was represented in
parallel ways. For both, start
points and end points were landmarks, paths, buildings, fields, intersections,
and the like. For maps, these were
often presented as icons, typically schematized as rough geometric shapes, and
often named, such as street or building names. Reorientations or turns were also schematized. In maps, they were typically portrayed
as lines that were more or less perpendicular. About half the participants used arrows to explicitly
indicate direction. Nearly half
used double lines to indicate paths, though single lines predominated. There are at least two ways to
interpret the use of double lines to indicate streets. The double lines could be iconic, as
streets have width. Alternatively,
they could indicate a perspective on the scene, conceiving of paths as planes
rather than lines. In directions,
there was a limited vocabulary and a limited structure, with slots for actions
(verbs), directions, and paths.
The common actions were the verbs "turn," "take a,"
"make a, Òand "go."
The verb was omitted in some descriptions, especially those that were
simply a list of the form: left on X, right on Y. Thus, in both maps and directions, changes of orientation
were schematized as turns of unspecified angles. In maps, they were depicted as approximate right angles
irrespective of their actual angle. Memory for intersections is also biased
toward right angles (e. g., Byrne, 1979; Moar and Bower, 1983; Tversky, 1981).
Progressions, too, were schematized.
In maps, they appeared either as straight lines or as slightly curved
ones, leaving out much of the detail in the real environment. The distinction between straight and
curved paths was also made in language.
By far, the two most frequent verbs for expressing progressions were
"go" and "follow," and they were used differentially for
straight and curved paths.
"Go" was used 17 times for straight paths and only twice for
curved (Chi-square = 9.0, p <.005), whereas "follow" was used only
5 times for straight paths but 20 times for curved ones (Chi-square=11.8, p
<.001). Thus, although the
actual paths and intersections had many forms, a single category of
intersection and two categories of path shape sufficed in schematization,
whether verbal or pictorial.
5.3 Sufficiency
in Descriptions and Depictions.
We found that both descriptions and
depictions consist of the same critical and supplementary information
schematized in similar ways. Was
the information sufficient for conveying the route? That is, did each segment contain all the essential components: start and end points, path, and
direction? For maps, the answer was a rousing yes. All of the maps contained all of the essential
information. For descriptions, in
contrast, the initial answer was no, much information was missing. In fact, 75% of the descriptions were
missing a start or an end point and 45% of the descriptions were missing
path/progression information.
However, many communications contain missing information that can be
inferred from context or medium (e. g., Clark and Clark, 1977). In the case of route directions, two
simple rules of inference allow recovery of most of the missing
information. The first is continuity.
According to continuity, if a start point is omitted, it is
assumed to be the same as the previous end point, or conversely, if an end
point is missing, it is assumed to be the same as the subsequent start
point. In fact, for depictions,
where continuity is inherent in maps, start and end points are not well-defined
or distinguishable. The second
inference rule is forward progression. According to forward
progression, the direction of motion is assumed to be forward. The first
protocol in Table 1 lacks any end points, yet they can be easily inferred from
the subsequent action. This
protocol has only one explicit mention of forward progression, at the end
("go down few miles"); rather, the forward progression is implicit.
After applying these two rules of inference, 86% of the directions were sufficient. In three of the descriptions, the direction of a turn was
missing and could not be recovered.
Although maps and directions schematize routes in similar
ways, maps are more complete than directions, which need to be supplemented
with inference rules. Another way
to put it is that directions are more schematized than maps. This difference, we believe, is
inherent in the graphic medium, in
the mapping of real space to representing space. Paths in real space are
continuous and forward moving given particular start and end points; they are
portrayed as such in representing space.
Even though the mapping from real space to representing space is
schematic rather than strictly iconic, it pragmatically presupposes the two
inference rules, continuity and forward progression. The naturalness of the mapping of space to space is further
evident in the greater variability of verbal expressions than pictorial
expressions for the four elements.
6 Schematization
of Space in Language and Cognition.
Clearly, there are parallels in the
way that cognition and language schematize the spatial world. Language is revealing in this
enterprise, not just closed-class terms, but open-class terms as well. Graphic communications, such as route
maps, are also schematized, again with similarities to language and
cognition. But both language and
cognition are rich, and are able to express and encode more or less
schematically, depending on the situation and how it is construed.
Figures, spatial relations among them, and paths between
them seem to be schematized similarly in language and cognition. Language serves well to convey routes
and environments, provided the routes and environments are well-known (e. g.,
Taylor and Tversky, 1992a, 1992b, 1996).
In contrast, language seems to be inadequate at conveying faces, voices,
and emotions. We can only
speculate on the answer. It is
likely that languages develop in small groups of people who know each other and
who use language in direct social encounters. Faces and voices are present in those situations so they
convey themselves--and emotions--directly, they do not have to be conveyed in words. They are tagged with proper names known
to the community so that, if needed, they can be gossiped about in their
absence. Proper names, in contrast
to category names and closed-class terms, do not convey any spatial information
in themselves. Like addresses or GPS coordinates, they point to individuals or
locations without giving any other information about them. Routes are described by terms with more
general spatial meanings. Unlike
faces and voices, they may not be present in the social encounters in which they
are discussed. Many cannot
even be viewed in entirety from a
single vantage point, much less the current one. What's more, individuals often set out alone to forage or
hunt, so that developing ways to communicate route directions is useful in
communal living. For faces,
voices, and routes, then, the Schematization Similarity Conjecture--that
language will be successful in communication to the extent that language and
'ception schematize similarly--receives support, along with a speculative
explanation.
At the outset we observed that perception inevitably affects
language, that at least in part, people develop vocabularies and syntax to
communicate about things in the world as they perceive them, and as they need
to talk about them. It is equally
clear that language influences perception. Language calls attention to particular things and states and
qualities and distinctions in the world and ignores others. Over repeated experience, the selective
attention encouraged by language can become habitual, so that it seems as if
language is no longer involved. Undoubtedly, habitual attention to certain
things, states, qualities, and distinctions in space affects the way space is
schematized, further intertwining the schematization of language and cognition.
References
1.
Biederman, I.: Recognition-by-components:
A theory of human image understanding. Psychological Review, 94, (1987) 115-147
2.
Bowerman, M.: Semantic and
syntactic development: A review of
what, when, and how in language acquisition. In: R. L. Schiefelbusch (ed.): Bases of language intervention, Vol. 1. University Park Press,
Baltimore. (1978a) 97-189
3.
Bowerman, M.: The acquisition of word meaning: An investigation into some
current conflicts. In: N. Waterson
and C. Snow (eds.): The development of communication: Social and pragmatic factors in language acquisition. Wiley,
NY. (1978b) 263-287
4. Brown,
R.: How shall a thing be called? Psychological Review, 65, (1958) 14-21
5. Bryant,
D. J., Tversky, B., & Franklin, N.: Internal and external spatial
frameworks for representing described scenes. Journal of Memory and Language,
31, (1992) 74-98
6. Bryant,
D. J., Tversky, B., and Lanca, M.: Retrieving spatial relations from observation
and memory. Unpublished manuscript. (1998)
7. Bruce,
V. : Recognizing faces. Erlbaum,
Hillsdale, NJ (1988)
8. Byrne,
R. W.: Memory for urban geography.
Quarterly Journal of Experimental Psychology, 31, (1979) 147-154
9.
Carlson-Radvansky, L. & Irwin, D. E.: Frames of reference in vision and
language: Where is above?
Cognition, 46, (1993) 223-244
10. Clark, E. V. What's in a word? On the child's
acquisition of semantics in his first language. In: T. E. Moore (ed.): Cognitive development and the
acquisition of language. Academic Press, NY. (1973). 65-110
11. Clark, H. H. & Clark, E. V.: Psychology and
language. Harcourt Brace Jovanovich, NY. (1977)
12. Clark, H. H. and Clark, E. V.: When nouns surface as
verbs. Language, 55, (1979), 420-477
13. Cutting, J. E., Proffitt, D. R., and Kozlowski, L.
T.: A biomechanical invariant for gait perception. Journal of Experimental Psychology: Human Perception and Performance (1978)
357-372
14. Denis, M.: La description d'itineraires: Des reperes pour des actions. Notes et Documents LIMSI No. 94-14 Juillet, 1994 (The description of
routes: Landmarks for actions)
(1994)
15. Diamond, R. and Carey, S.: Why faces are and are not
special: An effect of expertise.
Journal of Experimental Psychology: General, 115, (1986). 107-1227
16. Franklin, N., Henkel, L. A., and Zangas, T.: Parsing
surrounding space into regions. Memory
and Cognition, 23, (1995) 397-407
17. Franklin, N. and Tversky, B.: Searching imagined environments.
Journal of Experimental Psychology:
General, 119, (1990) 63-76
18. Garrod, S. C. and Sanford, A. J.: Discourse models
as interfaces between language and the spatial world. Journal of Semantics, 6, (1989) 147-160
19. Gentner, D.: Some interesting differences between
verbs and nouns. Cognition and Brain theory, 4, (1981) 161-178
20. Glenberg. A. M., Meyer, M., and Lindem, K.: Mental
models contribute to foregrounding during text comprehension. Journal of Memory
and Language, 26, (1987) 69-83
21. Gryl, A.: Analyse et modelisation des processus
discursifs mis en oeuvre dans la description d'itineraires. Notes et Documents LIMSI No. 95-30.
Decembre, 1995 (Analysis and modeling of discourse in the description of
routes) (1995)
22. Hagen, M. A. and Perkins, D.: A refutation of the hypothesis of the
superfidelity of caricatures relative to photographs. Perception, 12, (1983) 55-61
23. Hayward, W. G. and Tarr, M. J.: Spatial language and
spatial representation. Cognition,
55, (1995) 39-84
24. Hegarty, M.: Mental animation: Inferring motion from
static displays of mechanical systems. Journal of Experimental Psychology:
Learning, Memory, and Cognition, 18, (1992) 1084-1102
25. Herskovits, A.: Language and spatial cognition: An interdisciplinary study of the
prepositions in English. Cambridge, MA:
Cambridge University Press (1986)
26. Hochberg, J.: Perception. Prentice-Hall, Englewood
Cliffs, NJ (1978)
27. Hoffman, D. D. & Richards, W. A.: Parts of
recognition. Cognition, 18, (1984) 65-96
28. Howard, I. P.: Human visual orientation. Wiley, NY
(1982)
29. Huttenlocher, J. Hedges, L. V., and Duncan, S.:
Categories and particulars: Prototype effects in estimating spatial location.
Psychological Review, 98, (1991) 352-376
30. Huttenlocher, J. and Lui, F.: The semantic
organization of some simple nouns and verbs. Journal of Verbal Learning and Verbal Behavior, 18, (1979)
141-162
31. Ibbotson, A. and Bryant, P. E.: The perpendicular
error and the vertical effect.
Perception, 5, (1976) 319-326
32. Johansson, G.: Visual motion perception. Scientific
American, 232, (1975) 76-88
33. Kaiser, M. K., Proffitt, D. R., Whelan, S. M., and
Hecht, H.: Influence of animation on dynamical judgments. Journal of Experimental
Psychology: Human Perception and
Performance, 18, (1992) 669-690
34. Klein, W.: Local deixis in route directions. In: R. J. Jarvella and W. Klein (eds.):
Speech, place and action. Wiley, NY (1982) 161-182
35. Lakoff, G.: Women, fire and dangerous things: What categories reveal about the mind.
University of Chicago Press, Chicago (1987)
36. Landau, B. and Jackendoff, R.: "What" and
"where" in spatial language and spatial cognition. Behavioral and Brain Sciences, 16:2,
(1993) 217-238
37. Landau, B. Smith, L. B. and Jones, S. S.: The
importance of shape in early lexical learning. Cognitive Development, 3, (1988)
299-321
38. Landau, B. Smith, L. B. and Jones, S. S.: Syntactic
context and the shape bias in children's and adults' lexical learning. Journal
of Memory and Language, 31, (1992) 807-825
39. Leibowitz, H. W., Guzy, L. T., Peterson, E., &
Blake, P. T.: Quantitative perceptual estimates: Verbal versus nonverbal
retrieval techniques. Perception, 22, (1993) 1051-1060
40. Levelt, W. J. M.: Linearization in describing
spatial networks. In: S. Peters and E. Saarinen (eds.): Processes, beliefs, and questions.
Reidel, Dordrecht (1982) 199-220
41. Linde, C. and Labov, W.: Spatial structures as a site for the study of language and
thought. Language, 51, (1975)
924-939
42. Logan, G. D. and Sadler, D. D.: A computational
analysis of the apprehension of spatial relations. In: P. Bloom, M. A. Peterson, L. Nadel, and M. Garrett
(eds.): Language and Space. MIT Press, Cambridge, MA (1996) 493-529
43. Marr, D.: Vision: A computational investigation into
the human representation and processing of visual information. Freeman, NY
(1982)
44. Morrow, D. G., Bower, G. H., and Greenspan, S.:
Updating situation models during narrative comprehension. Journal of Memory and Language (1989)
292-312
45. Morrow, D. G. and Clark, H. H.: Interpreting words
in spatial descriptions. Language and Cognitive Processes, 3, (1988) 275-291
46. McBeath, M. K., Schiano, D. J., and Tversky, B.:
Three-dimensional bilateral symmetry bias in judgments of figural identity and
orientation. Psychological Science, 8, (1997) 217-223
47. Moar, I. & Bower, G. H.: Inconsistency in
spatial knowledge. Memory and Cognition, 11, (1983) 107-113
48. Morris, M. W. & Murphy, G. L.: Converging
operations on a basic level in event taxonomies. Memory and Cognition, 18,
(1990) 407-418
49. Murphy, G. L. and Smith, E. E.: Basic-level
superiority in picture categorization: Journal of Verbal Learning and Verbal
Behavior, 21, (1982) 1-20
50. Newtson, D., Hairfield, J., Bloomingdale, J., and
Cutino, S.: The structure of action and interaction. Social Cognition, 5,
(1987) 191-237
51. Palmer, S., Rosch, E. & Chase, P.: Canonical perspective
and the perception of objects. In: J. B. Long & A. D. Baddeley (eds.):
Attention and Performance, IX. Erlbaum: Hillsdale, New Jersey (1981)
52. Pani, J. R., William, C. T., and Shippey, G. T.:
Determinants of the perception of rotational motion: Orientation of the motion to the object and to the
environment. Journal of Experimental Psychology: Human Perception and
Performance, 21, (1995)
53. Nelson, T. O. and Chaiklin, S.: Immediate memory for
spatial location. Journal of Experimental
Psychology: Human Learning and Memory, 5, (1980) 212-228
54. Perrig, W. and Kintsch, W.: Propositional and
situational representations of text. Journal of Memory and Language, 24, (1985)
503-518
55. Rifkin, A.: Evidence for a basic level in event
taxonomies. Memory and Cognition, 13, (1985) 538-556
56. Rock, I.: The logic of perception. MIT Press,
Cambridge (1983)
57. Rosch, E.: Cognitive representations of semantic
categories. Journal of Experimental Psychology: General, 104, (1975) 192-233
58. Rosch, E.: Principles of categorization. In: E. Rosch and B. B. Lloyd: Cognition
and categorization. Erlbaum, Hillsdale, NJ (1978) 27-48
59. Rosch, E., Mervis, C., Gray, W., Johnson, D. and
Boyes-Braem, P.: Basic objects in natural categories. Cognitive Psychology, 6,
(1976) 382-439
60. Schooler, J. W. and Engstler-Schooler, T. Y.: Verbal
overshadowing of visual memories:
Some things are better left unsaid. Cognitive Psychology, 22, (1991)
36-71
61. Schwartz, D. L. & Black, J. B.: Analog imagery in mental model
reasoning: Depictive models. Cognitive Psychology, 30, (1996)
154-219
62. Shapiro, P. N. and Penrod, S.: Meta-analysis of
facial identification studies. Psychological Bulletin, 100, (1986) 39-156
63. Shiffrar, M. M. & Shepard, R. N.: Comparison of
cube rotations around axes inclined relative to the environment or to the
cube. Journal of Experimental
Psychology: Human Perception and
Performance, 17, (1991) 44-54
64. Spelke, E. S., Breinlinger, K., Macomber, J., and
Jacobson, K.: Origins of knowledge. Psychological Review, 99, (1992) 605-632
65. Streeter, L. A., Vitello, D. and Wonsiewicz, S. A.:
How to tell people where to go: Comparing navigational aids. International
Journal of Man-Machine Studies, 22, (1985) 549-562
66. Talmy, L.:
Semantics and syntax of motion.
In: John P. Kimball (ed.): Syntax and semantics, Vol. 4. Academic Press,
NY (1975) 181-238
67. Talmy, L.:
How language structures space.
In: H. L. Pick, Jr. & L. P. Acredolo (eds.): Spatial
orientation: Theory, research and
application. Plenum, NY (1983) 225-282
68. Talmy, L.: Lexicalization patterns: Semantic structure in lexical
forms. In: Language typology and
syntactic description, Vol. 3, T. Shopen (ed.): Grammatical categories and the
lexicon. Cambridge University Press, Cambridge (1985) 57-149
69. Talmy, L.: Force dynamics in language and cognition.
Cognitive Science, 12, (1988) 49-100
70. Talmy, L.: Fictive motion and change in language and
perception. In: P. Bloom, M. A. Peterson, L. Nadel, and M. Garrett (eds.):
Language and space. MIT Press, Cambridge (1996) 211-276
71. Taylor, H. A., Naylor, S. J. and Chechile, N. A.:
(In press) Goal-specific influences on the representation of spatial
perspectives. Memory and Cognition.
72. Taylor, H. A. & Tversky, B.: Descriptions and
depictions of environments. Memory and Cognition, 20, (1992a) 483-496
73. Taylor, H. A. & Tversky, B.: Spatial mental
models derived from survey and route descriptions. Journal of Memory and
Language, 31, (1992b) 261-292
74. Taylor, H. A. and Tversky, B.: Perspective in spatial
descriptions. Journal of Memory and Language, 35, (1996) 371-391
75. Taylor, M. M.: Effect of anchoring and distance
perception on the reproduction of forms.
Perceptual and Motor Skills, 12 (1961) 203-230
76. Tversky, B.: Distortions in memory for maps.
Cognitive Psychology, 13, (1981)
407-433
77. Tversky, B.: Categories and parts. In: C. Craig and
T. Givon (Eds.): Noun classes and categorization . John Benjamins Publishing
Co, Philadelphia (1985) 63-75
78. Tversky, B.: Distortions in memory for maps.
Cognitive Psychology, 13, (1981) 407-433
79. Tversky, B.: Distortions in cognitive maps.
Geoforum, 23, (1992) 131-138
80. Tversky, B. and Hemenway, K.: Objects, parts, and categories. Journal
of Experimental Psychology:
General, 113, (1984)
169-193
81. Tversky, B., Taylor, H. A., and Mainwaring, S.: Langage et perspective spatial (Spatial
perspectives in language). In: M.
Denis (Eds.): Langage et cognition spatiale. Masson, Paris (1997) 25-49
82. Vandeloise, C.: L'espace en francais: Semantique des prepositions spatiales.
Editions du Seuil, Paris. (1986)
83. Wunderlich, D. and Reinelt, R.: How to get there from here. In R. Jarvella and W. Klein (eds.):
Speech, place, and action. Wiley, Chichester (1982)183-201
84. Yin, R. K.: Looking at upside-down faces. Journal of
Experimental Psychology, 81, (1969) 141-145
85. Zacks, J. and Tversky, B.: What's happening? The structure of event perception. Proceedings of the Meetings of the Cognitive Science Society. Erlbaum, Mahwah, NJ (1997)