Guidelines for Experimental Design

In brief:
Design an experiment to test an interesting hypothesis about human cognition, inspired by the experiments and/or issues you hear and read about. Write up the experimental design in journal format. Briefly, the Introduction explains the hypothesis and justifies it by reference to prior research and by intelligent argument. The Method section describes the experimental subjects, stimuli, design, and procedure. The Results section explains how you would analyze the data, and the Discussion discusses possible outcomes and their implications.

Detailed advice:
Pick a simple idea that can be tested in a simple manner. One tried and true way to design an experiment is to alter an established experiment in an interesting way. You read about an experiment that you find interesting or puzzling. You wonder if the same technique would work if some of the conditions (for example, the task or stimuli) were different, or if a different technique would work if the conditions were the same. Or you think of another explanation for the phenomenon or theory, or a counter-example, a situation where you think the effect wouldn't hold. You read a few journal articles to see if your idea is feasible and to learn good techniques for designing an adequate experimental test. And there you have it! Of course, you still have to convice someone else that the idea you are proposing is valuable, usually by relating it to a broader theory or issue in cognition. Just as a previous experiment can serve as a model for a design, so a well-written article describing a similar experiment can serve as a model for your paper. Early on, consult with the instructor or one of the TAs about your ideas, to get feedback and relevant references. Your textbook is also a good source of relevant journal articles.

You are encouraged to work in groups. This way, you can bounce ideas for hypotheses and designs off each other. Research is basicallly a social phenomenon; even if you work alone, you do so in a community of scholars with shared interests and ideas. You are also encouraged to run your experiments, but if you decide to do that, you must check with one of us ahead of time. There are ethical and practical issues we can help you with.

Introduction:
You should have in mind a clear hypothesis that you want to test. However, you don't want to throw that idea at the reader first thing. You want to start out with a broader context, say the recent research on the problem, or a global analysis of the problem itself, or a theory, or a puzzle. That context should make your experiment look like a natural and interesting thing to do. It should be clear by now that personal justifications ("I thought it would be interesting to ..." or "I think that such and such is true...") are not appropriate. After you have described a context and presented and justified your hypothesis, the next step is to outline briefly how you plan to test it. This is the place to raise possible confounding factors and tell how you will control for them or to explain your choice of stimulus materials or subject population. You do not want to go into your procedures in detail because you do that in the next section. In the Introduction, you draw on other research wherever relevant.

Method:
This section is subdivided into subsections, such as:
Subjects, where you describe the number of subjects you used in each condition, and how they were selected
Design, where you describe succinctly the exact experimental design, including conditions and manipulations
Stimuli, where you describe the stimuli you used
Procedure, where you describe what happened in an experimental session from start to finish, usually including instructions given to subjects.
Not every experiment will need all of these sections, and some may need additional ones, such as
Equipment, where you would describe the equipment used.
In principle, the Method section should give enough information to allow another investigator to replicate your experiment. Some detail is important, like lengths of lists (and sometimes the complete list) or time allotted to each portion of the task, but some detail is not important, like the kind of room or type of slide projector.

Results:
Here, you describe the comparisons in the data that are critical to your hypotheses, for instance, that you expect the experimental group to outperform the control group on certain measures.

Discussion:
Here, you discuss the implications of findings, both the predicted ones and various unpredicted ones. What is their meaning, that is, what would they imply about the cognitive process under investigation? First discuss specific implications, and then more general ones. Again, relate your work/ideas to other work in the field.

References:
You list the articles/books you cite in your paper. Use the style in a recent journal article or in your text book as a model.

Answers to frequently asked questions. You are encouraged to collect some data, but it is not necessary. If you want to collect data, you must check with one of us ahead of time. As for length, brevity is a virtue in scientific writing. It would probably be difficult to say all that you need to say in less than 5-6 pages, but you probably do not need more than 10-12, depending, of course, on what you are proposing. If you are working in a group, you turn in one paper for the entire group.

Go back to the psyc 40 homepage.