Psych 224 - CS 329
Learning and Inference in Humans and Machines

Course information

Meeting time: Mondays and Wednesdays, 2:15-3:45
Location: 420-286 (Jordan Hall)
Instructor: Josh Tenenbaum
Email: jbt@psych.stanford.edu
Office hours: By email appointment (Mondays or Wednesdays after class are best times).

Course description

This course explores the connections between computational models and experimental studies of human learning and inference in several domains: visual perception, memory and information retrieval, supervised and unsupervised categorization, language acquisition, inductive reasoning, causal reasoning. Our twin aims are to reach a better understanding of human learning in computational terms and to bring computational systems closer to the capacities of human learners. Probabilistic models and statistical inference are unifying themes. The format will be an intensive focused discussion, with some informal lectures by the instructor or class participants on technical topics. Assignments include a wide range of readings, a few light computer exercises, and a final modeling project or paper. There are no formal prerequisites, but due to the advanced, interdisciplinary nature of this course, consent of the instructor is required for everyone enrolling. Having taken one or more of the following courses would be extremely helpful: Psychology 205, 210; Computer Science 221, 228, 229; Statistics 315A,B,C.

Syllabus


Discussion board (on the PanFora system)


Grading and Assignments

Class participation (including in-class presentation and discussion notes): 50%
Final project (including proposal): 50%

In addition to participating in class discussions and helping to lead discussion with at least one in-class presentation, students are required to submit brief discussion notes to the Discussion board (on the PanFora system) before each day's class. See here for details on this requirement.

For the final project, students are encouraged to work together on interdisciplinary topics that involve both experimental and modeling components. I will work with you to develop a promising project.

Readings

All readings on the syllabus fall into one of the following four categories:

- Readings listed on the syllabus as sections from Tom Mitchell's textbook Machine Learning. This book is recommended but not required for this course. All major readings from it will also be on reserve in the Math-CS library, if you decide not to purchase the book. If you do decide to purchase the book, buy it from Barnes and Noble, not Amazon or the Stanford bookstore. It is much cheaper at BN, and you can get a used copy if you want.

- Readings marked on the syllabus with [Reader] can be found in the printed course reader, available from the Stanford Bookstore. A copy of the reader will also be placed on reserve at the Math-CS library.

- Readings marked on the syllabus with [Web Link] can be found in digital format, with links off this web page below.

- Readings listed under "further reading" can be found on reserve in the Math-CS library, or on this web page below.

Readings available on the web:

Feldman, J. (2000) Minimization of Boolean complexity in human concept learning. Nature, 407 , 630-633. (pdf)

Shepard, R. N. (1987). Toward a universal law of generalization for psychological science. Science, 237 , 1317-1323. (pdf)

Tenenbaum, J. B. (1999). Rules and similarity in concept learning. (pdf)

Tenenbaum, J. B., Griffiths, T. L., and Ruml, W. (in preparation). A scaling law for psychological similarity. (html)

Tenenbaum, J. B., and Xu, F. (2000). Word learning as Bayesian inference. (pdf)

De Bonet, J. S., & Viola, P. (1997). Structure-driven image database retrieval. (pdf)

Ghahramani, Z. and Jordan, M.I. (1994). Learning from incomplete data MIT Center for Biological and Computational Learning Technical Report 108. (pdf)

Nigam, K., McCallum, A. K., Thrun, S., & Mitchell, T. (2000). Text classification from labeled and unlabeled documents using EM. (pdf)

Tenenbaum, J. B. & Griffiths, T. L. (submitted). The rational basis of representativeness. (pdf)

MacKay, D. J. C. (1995). Probable networks and plausible predictions -- a review of practical Bayesian methods for supervised neural networks. (pdf)

Smyth, P. (2000). Model selection for probabilistic clustering using cross-validated likelihood. (pdf)

Busemeyer, J., McDaniel, M. A., and Byun, E. (1997). The abstraction of intervening concepts from experience with multiple input-output causal environments. (pdf)

Ghahramani, Z. & Beal, M. J. (1999). Variational inference for Bayesian mixtures of factor analysers. (pdf)

Fraley, C. & Raftery, A. E. (1998). How many clusters? Which clustering method? Answers via model-based cluster analysis. (pdf only available from Stanford)

Gigerenzer, G., Czerlinski, J., & Martingon, L. (1999). How good are fast and frugal heuristics? (html)

Chater, N., Oaksford, M., Nakisa, R., & Redington, M. (under review). Fast, frugal, and rational: How rational norms explain behavior. (.html)

Saffran, J. R., Aslin, R. N., & Newport, E. L. (1996). Statistical learning by 8-month-old infants. (pdf only available from Stanford)

Brent, M. (1999a). Speech segmentation and word discovery: A computational perspective. (html)

Brent, M. (1999b). An efficient, probabilistically sound algorithm for segmentation and word discovery. (ps)

Marcus, G. F., Vijayan, S., Bandi Rao, S., & Vishton, P. M. (1997). Rule learning by seven-month-old infants. (pdf only available from Stanford)

Gomez, R. L. & Gerken, L. (1999). Artificial grammar learning by 1-year-olds leads to specific and abstract knowledge. Cognition, 70 , 109-135. (pdf)

Stolcke, A. & Omohundro, S. (1994). Inducing probabilistic grammars by Bayesian model merging. (pdf)

Redington, M. & Chater, N. (1998). Distributional information: A powerful cue for acquiring syntactic categories. (.pdf)

Charniak, E. (1991). Bayesian networks without tears. (.pdf)

Heckerman, D. (1996). A tutorial on learning with Bayesian networks. (pdf)

Tenenbaum, J. B. & Griffiths, T. L. (2001). Structure learning in human causal induction. NIPS 13. (pdf)

Gopnik, A., Glymour, C., & Sobel, D. (under review). Causal maps and Bayes nets: A cognitive and computational account of theory-formation. (.html)

Waldmann, M. & Martignon, L. (1998). A Bayesian network model of causal learning. (.pdf)

Lee, D. D. & Seung, H. S. (1999). Learning the parts of objects by nonnegative matrix factorization. (pdf only available from Stanford)

Tenenbaum, J. B., de Silva, V., & Langford, J. C. (2000). A global geometric framework for nonlinear dimensionality reduction. Science, 290 , 2319-2323. (pdf)

Roweis, S. T. & Saul, L. K. (2000). Nonlinear dimensionality reduction by locally linear embedding. Science, 290 , 2323-2326. (pdf)

Shepard, R. N. (2001). Perceptual-cognitive universals as reflections of the world. Behavioral and Brain Sciences, 24 (4). Section entitled ``Representations of an object's position, motion, and shape''. (html)

Tse, P. U. & Cavanagh, P. (2000). Chinese and Americans see opposite apparent motions in a Chinese character. Cognition, 74, B27-B32. (pdf)

Tenenbaum, J. B. & Freeman, W. T. (2000). Separating style and content with bilinear models. Neural Computation, 12 (6), 1247-1283. (pdf)

Landauer, T. K. & Dumais, S. T. (1997). A solution to Plato's problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. (html)

Steyvers, M. & Tenenbaum, J. B. (2001). Small worlds in semantic networks. (pdf)

Sloman, S. A., Love, B. C., & Ahn, W.-K. (1998). Feature centrality and conceptual coherence. (pdf)

Brin, S. & Page, L. (1998) The anatomy of a large-scale hypertextual web search engine. (html)

Strogatz, S. (2001). Exploring Complex Networks. Nature . (pdf from Stanford only)

Kleinberg, J. (1999). Authoritative sources in a hyperlinked environment. Journal of the ACM, 46 . (pdf)

Who wants to be a millionaire web searcher? (html)

Landauer, T. K., Laham, D., Rehder, B., & Schreiner, M. E. (1997). How well can passage meaning be derived without using word order? A comparison of latent semantic analysis and humans. (pdf) (see also other papers and demos at the LSA web site .)

Anderson, J. R., & Schooler, L. J. (1991). Reflections of the environment in memory. Psychological Science, 2 (6), 400-405. (pdf)

Gott, J. R. (1993). Implications of the Copernican principle for our future prospects. Nature, 363 , 315-319. (pdf)

Griffiths, T. L. & Tenenbaum, J. B. (2000). Teacakes, trains, toxins, and taxicabs: A Bayesian account of predicting the future. (pdf)

Redelmeier, D. A. & Kahneman, D. (1996). Patients memories of painful medical treatments: real-time and retrospective evaluations of two minimally invasive procedures. Pain, 66 , 3-8. (pdf)

Forster, M. (1999). Key concepts in model selection: performance and generalizability. J. Math Psych . (pdf)