Compared to What?

4 Jul 2008 - 3:48am
6 years ago
5 replies
239 reads
Jens Meiert
2004

Feeling free to blatantly point to my own take on "Compared to What?,"
<http://meiert.com/en/blog/20080703/compared-to-what/>. I understand
this concept's root in information design, but worry about the
supposed fact that not only many "designs", but rather news sources
fail in answering that question. While in turn, few people ask it
(which still reminds me of the situation that some people, users,
blame themselves for bad design/usability).

Curious about your thoughts.

--
Jens Meiert
http://meiert.com/en/

Comments

4 Jul 2008 - 4:44am
Steve Baty
2009

Jens,

Interesting article. I've posted a (rather lengthy, sorry) comment, but was
very happy to read your post.

Steve

2008/7/4 Jens Meiert <jens at meiert.com>:

> Feeling free to blatantly point to my own take on "Compared to What?,"
> <http://meiert.com/en/blog/20080703/compared-to-what/>. I understand
> this concept's root in information design, but worry about the
> supposed fact that not only many "designs", but rather news sources
> fail in answering that question. While in turn, few people ask it
> (which still reminds me of the situation that some people, users,
> blame themselves for bad design/usability).
>
> Curious about your thoughts.
>
> --
> Jens Meiert
> http://meiert.com/en/
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help
>

--
----------------------------------------------
Steve 'Doc' Baty B.Sc (Maths), M.EC, MBA
Principal Consultant
Meld Consulting
M: +61 417 061 292
E: stevebaty at meld.com.au

UX Statistics: http://uxstats.blogspot.com

Member, UPA - www.upassoc.org
Member, IA Institute - www.iainstitute.org
Member, IxDA - www.ixda.org
Contributor - UXMatters - www.uxmatters.com

4 Jul 2008 - 6:45am
Chauncey Wilson
2007

This is a very good issue Jens. One of the issues that happens
everyday is that someone on a product team will refer to poor
usability, but not highlight what the comparison is for this judgment
-- is it poor relative to the last version, user expectations, the
vice president's opinion, the major competitor..... In addition to
the object of comparison, there is also the common failure to clearly
express dimension of usability or user experience the comparison deals
with (learnability versus expert use/efficiency). So you could have a
dual problem where the object of comparison is not clear and the
dimension of interest is not explicit.

There is a book that deals with missing or inappropriate comparisons
in a book from 2001 (Chapter 4) that highlights the issue you wrote
about: Best, J. (2001). Damned lies and statistic. Bekeley, CA,
University of California Press. The book makes the point that it is
hard to understand or interpret lone statistics or implicit
comparisons. There are few books out like this one that highlight
this problem in media and scientific reports.

I routinely ask the questions "What is the goal of this (meeting,
project, brainstorming session...)?, but you bring up the point that
when a person makes a comparative statement, that we should ask what
the object of comparison is.

Another twist on this issue would involve some of the cognitive biases
that can affect how we perceive data. Some of the work by Kahneman and
Tversky and others about common, but powerful cognitive biases like
the "availability heuristic" where people judge the liklihood of
things by their vividness (which overrides baseline statistical
information) and the law of small numbers.

Very good point and nice article.

Chauncey

On Fri, Jul 4, 2008 at 4:48 AM, Jens Meiert <jens at meiert.com> wrote:
> Feeling free to blatantly point to my own take on "Compared to What?,"
> <http://meiert.com/en/blog/20080703/compared-to-what/>. I understand
> this concept's root in information design, but worry about the
> supposed fact that not only many "designs", but rather news sources
> fail in answering that question. While in turn, few people ask it
> (which still reminds me of the situation that some people, users,
> blame themselves for bad design/usability).
>
> Curious about your thoughts.
>
> --
> Jens Meiert
> http://meiert.com/en/
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help
>

5 Jul 2008 - 5:32am
Jens Meiert
2004

> > <http://meiert.com/en/blog/20080703/compared-to-what/>

Thank you very much, Steve, Chauncey! (Will reply in the post as well soon.)

> One of the issues that happens
> everyday is that someone on a product team will refer to poor
> usability, but not highlight what the comparison is for this judgment
> -- is it poor relative to the last version, user expectations, the
> vice president's opinion, the major competitor..... In addition to
> the object of comparison, there is also the common failure to clearly
> express dimension of usability or user experience the comparison deals
> with (learnability versus expert use/efficiency). So you could have a
> dual problem where the object of comparison is not clear and the
> dimension of interest is not explicit.

Absolutely, and this seems to be all too common.

> There is a book that deals with missing or inappropriate comparisons
> in a book from 2001 (Chapter 4) that highlights the issue you wrote
> about: Best, J. (2001). Damned lies and statistic. Bekeley, CA,
> University of California Press.

Thank you, saved. Looks like a nice complement to Tufte's remarks on
that matter.

> I routinely ask the questions "What is the goal of this (meeting,
> project, brainstorming session...)?, but you bring up the point that
> when a person makes a comparative statement, that we should ask what
> the object of comparison is.

Exactly. And I really see that a lot on news, even (or especially)
when they just state that "women cause fewer traffic accidents than
men" and then don't offer any numbers on how many women and men
actually participate in traffic, and the meaning of these statements
might change dramatically if there's a ratio of 1:10, 1:3, 1:1, or
vice versa (so while recipients might think "women are safe drivers",
the exact opposite might be the case).

> Another twist on this issue would involve some of the cognitive biases
> that can affect how we perceive data. Some of the work by Kahneman and
> Tversky and others about common, but powerful cognitive biases like
> the "availability heuristic" where people judge the liklihood of
> things by their vividness (which overrides baseline statistical
> information) and the law of small numbers.

Which is an interesting observation, indeed.

Best,
Jens.

--
Jens Meiert
http://meiert.com/en/

5 Jul 2008 - 5:46am
Steve Baty
2009

You might also like to take a look at Stephen K. Campbell's Flaws and
Fallacies in Statistical Thinking, Dover Publications, 2004. It is less
social studies-oriented and more focused on the sorts of error in
interpretation & reporting that I mentioned in my comments.

Cheers
Steve

2008/7/5 Jens Meiert <jens at meiert.com>:

> > > <http://meiert.com/en/blog/20080703/compared-to-what/>
>
> Thank you very much, Steve, Chauncey! (Will reply in the post as well
> soon.)
>
>
> > There is a book that deals with missing or inappropriate comparisons
> > in a book from 2001 (Chapter 4) that highlights the issue you wrote
> > about: Best, J. (2001). Damned lies and statistic. Bekeley, CA,
> > University of California Press.
>
> Thank you, saved. Looks like a nice complement to Tufte's remarks on
> that matter.
>
>

5 Jul 2008 - 8:25am
Jim Hoekema
2004

Jens,

Excellent point, and some great responses already.

It seems to me there was a soul/R&B song out at least 20 years ago
with the title "Compared to What?" Good song, too.

I'll only add that people commit the fallacy of not adding a
comparative reference when they say that an interaction or any design
element should be "consistent" or "intuitive." The same ambiguity is
there - compared to the other buttons on this page? Compared to other
screens in this program? Compared to other experiences by the same
audience in other contexts altogether? Often you have a great deal of
flexibility, depending on what you decide to be consistent with!

- Jim Hoekema

On Jul 4, 4:48 am, "Jens Meiert" <j... at meiert.com> wrote:
> Feeling free to blatantly point to my own take on "Compared to What?,"
> <http://meiert.com/en/blog/20080703/compared-to-what/>.
> Jens Meierthttp://meiert.com/en/

Syndicate content Get the feed