« "MetroNerd" - a mobile phone app to make nerds cooler | Main | How the Japanese interpreted "captology" »
June 05, 2006
Our web credibility work praised for vagueness
In the current issue of ACM's Interactions, IBM's Fred Sampson writes two pages about how metrics can get in the way of real insight.
(This is Fred Sampson, who works with IBM, STC, and DUX.)
While our Stanford lab focuses a lot on measurements, Fred praises part of our work where we don't go mega-quantitative. In one large (quantitative) study, we found that people mostly evaluated the credibility of web sites based on the "design look." We gave examples of this concept, but we didn't go much farther on this point. After we published that research many people have asked us to break "design look" into parts they can study and manipulate. Now, to my surprise, Fred takes the contrary stance: "Please don't insult me further by trying to quantify looking good."
If you know my work, you know I love categorizing and analyzing. I thrive on breaking complex ideas into their simplest parts. However, in this particular case, I agree with Fred. But I agree for reasons that may surprise you. The web cred study he cites explores over 40 common-sense concepts. If we had quantified each one, we would have either (a) done a terrible job or (b) never completed the study. Even operationalizing the concept of "design look" is a big task, probably about the size of a dissertation to get a preliminary answer (any takers?)
Here's an excerpt from the Fred Sampson's full article:
I am gratified to see that Stanford researcher B.J. Fogg and colleagues, when evaluating the credibility of Web sites, count user responses but do not try to quantify aspects such as "design look" or "information focus." Credible Web sites look good, just as we know that tall, attractive people are somehow seen as more credible than the rest of us. See, for instance, Malcolm Gladwell on the predominance of tall CEOs, and the presidential good-looks (and dismal performance) of President Warren Harding. But please don’t insult me further by trying to quantify looking good; we all know that there are no tens.
-- BJ Fogg
Posted by BJ Fogg at June 5, 2006 09:08 AM