What kinds of question CANNOT get answered?
1. Predictions, recollections, and impressions
are WEAK
Forrester data on MOST LIKELY TO BUY (= 15% in one year)
Memory of an event suffers numerous psychological biases
Vague impressions cannot predict market utility
(e.g., "How would you like to receive unsolicited obscene messages every
day?")
2. Attitudes fail to strongly correlate with behavior
What's the most powerful predictor of food donations?
What do years of social scientific research reveal about attitudes?
3. Rare events, no matter how profitable, cannot always be simulated
4. Weak effects (and many design decisions) cannot really
be surveyed
5. Complexity of information architecture cannot be done experimentally
How many different combinations need to be
tested when you have 7 components that can be laid out independently? (Applies
to scheduling and menu design)
5,040 distinct cells (multiply by data points needed per cell).
Grows to 3,628,800 with 10 independent items
With 63 independent variables, the options exceed the # of atoms in the universe