back to syllabus
Lecture Notes - 07/15/99 Seeing in .
- Note regarding the reading (Chapter 7): Skip the section called "Disparity from spatial phase", and don't worry about any mentions of "filters" in this chapter. It reads fine if you ignore those few references.
- In this lecture, we'll continue talking about our ability to perceive "where", although we'll be heading away from the M-pathway and talking more about 'perception'.
1. Basic issues and terms in depth perception
- Recovering 3 dimensions: the physical world is 3-dimensional, but the light that reflects off of the objects in the world project to a 2-dimentional image on the retina. Somehow, we must use information in the 2-D image to "recover" or "reconstruct" a 3-D percept.
- Monocular and Binocular information: while most people nowadays know that having 2 eyes is very helpful for depth perception, you can still do a pretty good job of perceiving depth with only 1 eye. We'll discuss the monocular ("one eye") and binocular sources of depth information.
- Extraretinal (or "physiological") sources of depth information: even more so than in motion perception, the movements and positions of the eyes will contribute important information to depth perception.
- Sometimes stereo vision is called stereopsis, which is actually Greek for "solid sight". That's important: stereo doesn't mean "binocular"; it means "solid", like the objects in the world.
2. Depth cues
- In class, I'll ask you to come up with examples of monocular and binocular cues to depth. Before class, try thinking of how you could judge depth with only 1 eye. Then, try to come up with several binocular cues. After class, feel free to review the list of depth cues.
- Remember that each eye has a slightly different view of the world, by virtue of their offset positions in the head. However, the 2 views overlap considerably; this doesn't happen in all animals (think horses, fish, etc.). Why would we have such a redundant (overlapping) coverage of an incomplete (smaller than many animals') field-of-view ("FOV")?
- It's likely that we sacrifice a 360 deg FOV because multiple, but slightly different, views from the 2 eyes provides us very important information about depth. Because our eyes are offset horizontally, the 2 eyes' images are shifted horizontally relative to each other; this difference is called the (horizontal) disparity.
- Thumb demo revisited: remember that when you hold your thumb close to your face, and alternatingly open and close your right and left eyes, your thumb appears to jump horizontally. If your eyes weren't offset side-by-side, but instead were one-above-the-other, what type of disparity would occur, and what would happen to the image of your thumb in this demo?
- Horopter: When you look an an object at some depth, you position our eyes so that the image it produces on both retinas correspond (i.e., zero disparity). The horopter is the imaginary 3D surface that extends from this object to include all other points at which the images fall onto corresponding places in both eyes.
- Uncrossed disparity: an object farter away than the horopter has uncrossed disparities-- you'd need to 'uncross' your eyes to fixate on it.
- Crossed disparity: an object closer than the horopter has crossed disparity-- you'd need to 'cross' your eyes to fixate on it.
4. Stereograms and other 3D media
- People (and popular culture) have long been fascinated with the artificial creation of 3D percepts. Wheatstone's original (1838) stereoscope was the first example of this. It presented an image to each eye separately; while the images were of the same thing, they differed just as they would if you were really looking at a 3D object with actual depth (instead of a flat sheet of paper). By artificially including disparity in the pair of images, people looking through a stereoscope could see objects in depth.
- Another way of introducing disparity using a 2D surface is the technique used in old 3D movies: red-green anaglyphs. By wearing red/green glasses, you can superimpose a red and green image on the screen, but still present different views to each eye. If the two views can differ as above, they can produce a percept of depth. Trouble: can't do color with this technology.
- Random dot stereograms: Invented by Bela Julesz, these are very convincing evidence that disparity (alone) can produce strong sensations of depth.
- An example: A common way to "fuse" the b&w stereogram is to hold your finger in your field-of-view between your eyes and the stereogram. Fixate on your finger, not on the stereogram. By doing this, you'll see multiple images of the stereogram. Move your finger until the images fuse; you know you've got it right when (1) you can see a square floating above the surface of the dots; and (2) you have a (monocular) "ghost" image on each side of the stereo image. Note: I used to have a lot of trouble seeing these until I picked up this trick: If the finger-strategy doesn't work for you, hold a transparent piece of glass or plastic between you and the image. Move the plastic sheet (I find that the clear front of a CD case works well) back and forth, slowly, between you and the image until it jumps out in depth. Make sure to keep fixated on the transparent surface: it often helps to use a surface with a slight mark or scratch to do this.
- Second, see how to make a random-dot stereogram: (1) Place a bunch of dots randomly in a square region. Make 2 copies. In one copy, shift the central square region to the left; in the other, shift the central square region to the right. Fill in the open sots left over from the shift with more random dots. Now, think about how this simple process mimics the differences between the views of real 3D objects.
- Autostereogram: also know as "magic eye" displays (yes, the same as you see nowadays in calendars and posters), autostereograms fit two slightly different images into the same region by hiding them in repetitive patterns. A very simple example of this is the wallpaper illusion-- if you stare at striped or repetitive wallpaper for a while, sometimes you can fixate one eye on one stripe and the other eye on a different stripe. The wallpaper then appears to jump out in depth. This is often difficult, as it requires you to fixate on a point that is behind the picture, while focusing your eye to the picture itself. Note: if you're having trouble seeing these, try leaning very close to the image, so that you can no longer fuse it (i.e., so that you get double-vision). Now, try to get the double images to match up. Then, slowly move backward and try to keep the "match". A couple of tries usually gets even the hardest magic eyes.
5. Correspondence problem
- We've talked about disparity as deriving from the different relative positions of images in the two eyes. But in order to calculate the offset (disparity), the brain must know which parts of each image to compare. In other words, there is a correspondence problem.
- Sherrington (1906) proposed that each eye's image gets processed so that higher-level forms can be matched. This technique, while easy to intuit, should seem somewhat improbable-- why would the visual system do everything twice?
- Austin (1907) publicized a phenomenon that provided early counterevidence against Sherrington's view: when different faces are presented to each eye, the faces blend, yielding a single and novel face. Often, the face is more attractive than either of its components. Wow.
- Julesz's random-dot stereograms (see above) posed stronger challenges to both of these theories. Remember that these random-dot patterns have no identifiable forms to match, but are somehow fused.
- Panum's fusional area: When you fixate on an object at a certain depth, all other objects along the horopter, plus and minus a small amount, will also appear "fused". Objects outside this region (Panum's fusional area) will appear as double images.
- Diplopia: double vision. Demonstrate this simply by holding your thumb near your eyes and looking at something far. This is constantly occurring, in less extreme cases (i.e., objects further from your eyes than in the thumb demo), but you tend not to notice it. Why? (see text for some possibilities).
- Binocular rivalry: an example of when the visual system can't solve the correspondence problem. In rivalry, very different images are presented to each eye. Instead of a blending, one eye's view dominates for a while, then the other eye's view dominates. Occassional mixes during transition periods.
6. Stereovision in the brain
- Binocular neurons. The neurons in the brain that are responsible for figuring out disparity must receive inputs from both eyes. So, we know that retinal cells (photoreceptors, retinal ganglion cells) cannot perform this. Likewise, LGN neurons are organized in alternating layers of eye-of-origin. And, although we talked about ocular-dominance columns in V1, there are also some binocular neurons: neurons that receive input from both eyes. Note that after V1, all visual neurons are binocular.
- Disparity-selective neurons: some V1 neurons (and many in V2 and MT, among other areas) fire only when a line passed through their receptive field has the proper orientation, direction, and disparity.
- See the distribution of preferred disparities of many cells in V1, notice that many neurons prefer zero disparity.
- Stereoblindness. 5-10% of people are stereoblind. Some people are totally stereoblind, while others are just blind to crossed or uncrossed disparities. Ine major cause of stereoblindness is strabismus, or "wandering eye". If left untreated past infancy, this can lead to amblyopia, or cortical blindness (a general term for blindness when the optics and structure of the eye and retina are alright).
7. Judging size and shape
- A fundamental of object perception is that the objects we see are in a 3D world. Therefore, we are very adept at transforming 2D images (like drawings) into 3D percepts. In the case of the Shepard Tables illusion, we cannot perceive the two rectangles as the same (merely rotated) because we cannot avoid perceiving them as tabletops viewed in depth (which would not be the same size).
- Size constancy. The visual system compensates for perspective and distance when judging size. While we all know that objects (or people) who are far away yield a smaller retinal image than nearby things, the amount of compensation that our visual systems automatically perform is often surprising.
- Shape constancy. Often, we view objects from an angle. The retinal image is accordingly slanted, but we don't really perceive much of this distortion. View this picture of Richard Nixon standing in front of a Richard Nixon poster. Note that the photographer was at an angle from the poster, and notice that the poster-Nixon looks distorted (slanted). Now, if you turn yourself so that you are viewing the picture from an angle, notice that the "real" Nixon doesn't appear so distorted. Why?
- Perspective in art
- Refer back to the list of depth cues and note that many of them are standard tools employed by artists to produce percepts of depth.
- However, there are also large stylistic differences over time and cultures in the representation of depth. Consider these examples and think about what they rely on to produce depth (and how effective they are):