Sunday, November 14, 2004

The Preference Problem

Malcolm Gladwell, a writer for the New Yorker, gave a presentation on understanding the instability of preference. This is going to be part of the topic of his upcoming book - Blink - The Power of Thinking without Thinking. Companies spend millions doing consumer testing and focus groups - most movies are tested and edited based on focus groups. Primaries and Presidential elections are, in large part, decided on the basis of telephone polls. Gladwell takes a step and tries to establish to what extent people are accurately predicting their consumer preference when they participate. Gladwell recounts the story of the Herman Miller Aeron chair. First he details how Herman Miller looked at the various problems with existing chairs and how they came up with innovative solutions. Then they ran consumer testing - when it came to comfort, the chair received very very high scores. However when people looked at how attractive they thought the chair was, the chair scored remarkably poorly. In fact, when they showed the chair to office managers, the overwhelming response that was it was far too ugly. But over a series of years the chair sold and went on to become one of the most popular selling chairs ever. Moreover, when they did studies later asking people how they liked the appearance of the chair the results had changed - now people found the chair not just inoffensive but actually attractive. Preferences are unstable. People think that preferences are immutable but in fact they are unstable. As a further example, Gladwell looks at the series of events that caused Coke to introduce New Coke. Despite the fact that Coke was the #1 drink ahead of Pepsi, Pepsi ran a remarkably successful ad campaign called the Pepsi Challenge. Basically people in malls were given samples of Coke and Pepsi without being informed of which is which and they were asked to pick which they preferred. Overwhelmingly they said that the preferred Pepsi. What could be more unbiased than a blind taste test? So Coke launched a campaign to produce a drink that would do well in the blind taste test. The product was New Coke and it was preferred over Pepsi in a blind taste test ~ 55% / 45%. The product was introduced and went on to become a business school case study in product launch disasters. Much has been made of bad marketing or lack of market place acceptance but Gladwell thinks the answer is a lot clearer than that. The problem has to do with the instability of preference. The principal problem with the Pepsi Challenge test is that you only take a sip. It turns out that if you are sampling drinks and you only take a sip you will invariably pick the sweeter sample. But a sweeter drink that is more palatable for a sip becomes revolting when you drink a whole glass. This was the genius of the Pepsi Challenge - Pepsi knew that they had a sweeter drink so they developed a test (by design or not) that would favor their drink. There is another type of test called "home use" where the participant is given a case or two of the product to use and it invariably comes up with different results than a sip test. Gladwell goes on to cite different ways in which focus groups can be deceived:
  1. The Triangle problem: The Triangle problem shows the instability of preference - start with two samples (Coke, Pepsi) and pick which one is one. People can do this correctly about 80% of the time. However, if you give people 3 samples of two drinks and ask them to determine which two are alike it is significantly more difficult - in fact people can do this correctly 33% of the time which is the same as randomly guessing. Why believe people when they say they don't like something if preference can be this unstable?
  2. The Story Telling Problem: Many times when people tell you why they like things they are just making up stories. Gladwell cites an experiment where top tennis players were asked how they hit a top-spin forehand serve. Without exception, they all said that they rolled their wrist at the moment of impact. However when analyzed with a high speed camera, none of them actually did this. In fact, if you roll your wrist at the moment of impact it's very difficult to hit the serve. Despite the fact that all of the people they asked are masters at hitting this type of serve, when asked how they do it they are incapable of introspecting what they do and resort to telling a story (most likely how they were told to do it).
  3. Perils of Introspection: The Act of getting someone in a room and asking them to explain their preference for something can affect the answer that they are likely to give you. Gladwell cites an experiment where two groups of people were asked to select a poster from a set. However the second set of people were asked to explain why they picked their poster. After 6 months, the people were asked how they liked the poster they had picked. Overwhelmingly, the people who were asked to explain their selection were unhappy with the poster whereas the first group of people were very happy with their selection. Moreover, the first group of people were more likely to pick impressionist painters whereas the group that had to explain their selection were more likely to pick kitten posters. In this case, forcing people to explain why they liked a poster forced them to like a poster that they could easily explain why they liked it. This is a pretty big problem if you are trying to understand why people like one product over another.
Can we just not trust people when it comes to product testing? Gladwell thinks that they're are a class of products for which consumer testing work - somethings are ugly and will always be ugly. But some products challenge people and they don't know what to think about them. If you are coming up with something that is new and radical and challenging then you need to be wary of feedback and consumer testing as an absolute science. When you ask people what they think of things and they respond with words like "no" and "ugly" and "I don't like that" you have to take it with a very large grain of salt. People are very bad at introspection and consumer testing is treated too much like a science.

0 Comments:

Post a Comment

<< Home