Most usable doesn't always mean best solution

3 Mar 2008 - 2:46pm
6 years ago
28 replies
586 reads
Todd Warfel
2003

Last week we were conducting some research for a client. The research
included testing of a registration form for their product/service.
During registration, participants filled out their contact info (e.g.
name/email) then were directed to a security screen, where they were
prompted to answer three security questions—pretty standard for
financial systems these days (this was not a financial system, but was
for backing up all your stuff on your computer through their service—
so equally important to individuals).

What surprised us was that the corresponding drop down menu for each
security item had 20 questions in each—a very long list. We had some
initial concerns that participants would find this overwhelming, a bit
tedious, and my be put off by the whole thing. What we found was quite
the opposite, actually. Every participant 11/11 felt reassured with
the long list of questions. Responses included:
"This makes me feel more safe."
"These questions are harder to break. Everyone can guess my eye color,
but not my favorite flavor of ice cream."
"This is good. Lots of questions are harder to break."

And then there were the "Well, I wouldn't pick favorite color or ice
cream. Those things change too often depending on my mood."
Incidentally, this was heard from 4 women, but none of the men. Just
an observation folks, don't shoot the messenger.

The point is that technically, this form wasn't more usable—it was in
fact less usable, took more effort, and time to complete than if there
were only say 5 questions in each menu. 11/11 participants rated this
as being a 1=very easy and high satisfaction, which goes to show you
that the most usable solution isn't always the best solution.

Cheers!

Todd Zaki Warfel
President, Design Researcher
Messagefirst | Designing Information. Beautifully.
----------------------------------
Contact Info
Voice: (215) 825-7423
Email: todd at messagefirst.com
AIM: twarfel at mac.com
Blog: http://toddwarfel.com
----------------------------------
In theory, theory and practice are the same.
In practice, they are not.

Comments

3 Mar 2008 - 2:54pm
Elizabeth Buie
2004

Todd Zaki Warfel writes:

>The point is that technically, this form wasn't more usable—it was in
>fact less usable, took more effort, and time to complete than if there
>were only say 5 questions in each menu. 11/11 participants rated this
>as being a 1=very easy and high satisfaction, which goes to show you
>that the most usable solution isn't always the best solution.

Ah, but your description considers usability only in terms of efficiency.
When you look at the definition of usability as given, for example, in
ISO 9241, you will see that it includes effectiveness and -- key to this
point -- satisfaction. In this light, the solution you found was indeed
the most usable.

Elizabeth

3 Mar 2008 - 3:12pm
Todd Warfel
2003

On Mar 3, 2008, at 2:55 PM, Z S Zaiss wrote:

> I'd still say your results prove usability - just not efficiency

The main point here is that usability doesn't always equate to
efficiency, to rephrase the subject line. A purest theoretical
perspective on usability is that usability=efficient, which is not
necessarily the case in reality.

Cheers!

Todd Zaki Warfel
President, Design Researcher
Messagefirst | Designing Information. Beautifully.
----------------------------------
Contact Info
Voice: (215) 825-7423
Email: todd at messagefirst.com
AIM: twarfel at mac.com
Blog: http://toddwarfel.com
----------------------------------
In theory, theory and practice are the same.
In practice, they are not.

3 Mar 2008 - 3:14pm
Caryn Josephson
2004

There's a second piece to this, though. For the 'ice cream' type of
questions, 9 months later when they've forgotten their password, will they
be able to successfully re-set it, or will they have forgotten the
'answer'?

>Last week we were conducting some research for a client. The research
>included testing of a registration form for their product/service.
>During registration, participants filled out their contact info (e.g.
>name/email) then were directed to a security screen, where they were
>prompted to answer three security questions—pretty standard for
>financial systems these days (this was not a financial system, but was
>for backing up all your stuff on your computer through their service—
>so equally important to individuals).
>
>What surprised us was that the corresponding drop down menu for each
>security item had 20 questions in each—a very long list. We had some
>initial concerns that participants would find this overwhelming, a bit
>tedious, and my be put off by the whole thing. What we found was quite
>the opposite, actually. Every participant 11/11 felt reassured with
>the long list of questions. Responses included:
>"This makes me feel more safe."
>"These questions are harder to break. Everyone can guess my eye color,
>but not my favorite flavor of ice cream."
>"This is good. Lots of questions are harder to break."
>
>And then there were the "Well, I wouldn't pick favorite color or ice
>cream. Those things change too often depending on my mood."
>Incidentally, this was heard from 4 women, but none of the men. Just
>an observation folks, don't shoot the messenger.
>
>The point is that technically, this form wasn't more usable—it was in
>fact less usable, took more effort, and time to complete than if there
>were only say 5 questions in each menu. 11/11 participants rated this
>as being a 1=very easy and high satisfaction, which goes to show you
>that the most usable solution isn't always the best solution.
>
>Cheers!
>
>Todd Zaki Warfel
>President, Design Researcher
>Messagefirst | Designing Information. Beautifully.
>----------------------------------
>Contact Info
>Voice: (215) 825-7423
>Email: todd at messagefirst.com
>AIM: twarfel at mac.com
>Blog: http://toddwarfel.com
>----------------------------------
>In theory, theory and practice are the same.
>In practice, they are not.
>
>________________________________________________________________
>Welcome to the Interaction Design Association (IxDA)!
>To post to this list ....... discuss at ixda.org
>Unsubscribe ................ http://www.ixda.org/unsubscribe
>List Guidelines ............ http://www.ixda.org/guidelines
>List Help .................. http://www.ixda.org/help
>

3 Mar 2008 - 3:17pm
Todd Warfel
2003

On Mar 3, 2008, at 3:14 PM, Caryn Josephson wrote:

> For the 'ice cream' type of questions, 9 months later when they've
> forgotten their password, will they be able to successfully re-set
> it, or will they have forgotten the 'answer'?

In our study, every participant avoided those questions for that very
fear. However, that would definitely be an aspect to follow up on.
That scenario wasn't part of our initial test and wasn't built into
the beta.

Cheers!

Todd Zaki Warfel
President, Design Researcher
Messagefirst | Designing Information. Beautifully.
----------------------------------
Contact Info
Voice: (215) 825-7423
Email: todd at messagefirst.com
AIM: twarfel at mac.com
Blog: http://toddwarfel.com
----------------------------------
In theory, theory and practice are the same.
In practice, they are not.

3 Mar 2008 - 4:41pm
dmitryn
2004

It would be interesting, too, to see whether the types of questions
users perceive as memorable really do prove memorable some time down
the road. Is anyone aware of research results that are relevant to
this question?

Dmitry

On Mon, Mar 3, 2008 at 12:17 PM, Todd Zaki Warfel <lists at toddwarfel.com> wrote:
>
> In our study, every participant avoided those questions for that very
> fear. However, that would definitely be an aspect to follow up on.
>
>
> Cheers!
>
> Todd Zaki Warfel
> President, Design Researcher
> Messagefirst | Designing Information. Beautifully.
> ----------------------------------
> Contact Info
> Voice: (215) 825-7423
> Email: todd at messagefirst.com
> AIM: twarfel at mac.com
> Blog: http://toddwarfel.com
> ----------------------------------
> In theory, theory and practice are the same.
> In practice, they are not.
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help
>

3 Mar 2008 - 5:21pm
Becubed
2004

> which goes to show you
> that the most usable solution isn't always the best solution.

By far my favorite example of this maxim is automatic vs. standard
transmissions in vehicles. Imagine doing a usability test of these two
technologies: the automatic would win, hands-down.

Yet so many people *passionately* prefer standard and wouldn't be caught
dead driving an automatic.

3 Mar 2008 - 5:26pm
Elizabeth Buie
2004

Robert Barlow-Busch writes:

>By far my favorite example of this maxim is automatic vs. standard
>transmissions in vehicles. Imagine doing a usability test of these two
>technologies: the automatic would win, hands-down.

No, the automatic would come out as the most learnable. For those who are
familiar with it, it is also the most usable. For example, I have never
owned an automatic, and I always feel clumsy when I am forced to rent one.
But I'm glad I learned on one, lo! those many years ago.

Elizabeth

3 Mar 2008 - 5:30pm
Robert Hoekman, Jr.
2005

> There's a second piece to this, though. For the 'ice cream' type of
> questions, 9 months later when they've forgotten their password, will they
> be able to successfully re-set it, or will they have forgotten the
> 'answer'?

Great point!

One of the trickiest aspects is remembering the exact casing and such of the
answer you entered. Every so often, my bank's site asks me the model of my
first car, and I have to remember whether or not I included make and model,
or just model, whether I capitalized it or not, etc.

A system that was forgiving in this area, like the IxDA sign-in process,
could be great in some situations.

-r-

3 Mar 2008 - 5:33pm
Brandon E.B. Ward
2008

I think the 'standard vs automatic' is very much akin to Linux/Unix user vs Regular user.

I think everyone could agree that automatic is technically 'easier', but you lose the ability to have fine-tune control if/when you want it like you've got w/ a stick. The people I know that drive stick do so because driving is a passion for them and they don't trust a machine to decide when/where/how to shift for them. They also like to work on their own cars, tweek them, mod them etc.

Power-users and people who like tinkering w/ the fiddly-bits of their OS, trying new things, getting under the hood of the OS, using OpenSource apps, also like this ability in their computers. Setting up and using Linux/Unix etc. may not be quite as 'easy' as just installing OS X or Windows, and finding apps, settings etc. might be a bit more of a challenge, but those who choose either camp do so for obvious reasons, foregoing either some of the options or control they might get for the ability to do things more quickly/easily.

Sorry if this isn't very coherent - but I think my point is in there somewhere.

B

3 Mar 2008 - 7:48pm
Rob Tannen
2006

Todd - Depending on the specifics of the question choices, it's not
clear whether more questions is not in fact more efficient. One of
the issues with "challenge questions" is that users may not have
appropriate or memorable choices to select from. For example, your
high school didn't have a mascot or your dad doesn't have a middle
name.

This is especially noticeable with an international user base who
have less in common culturally to draw from. Therefore, more
question choices increases the likelihood that there there are
questions which users can readily come up with an answer for, rather
than a limited choice where users struggle to find appropriate
questions. In other words, more questions is more efficient from a
mental task completion view, versus a speed of reading perspective.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=26755

3 Mar 2008 - 8:51pm
Jennifer Berk
2007

I have to agree with this, and also mention that poorly chosen
"challenge questions" may be viewed as security problems. I came
across a (not needing serious security, i.e. not financial or similar)
site recently where one of the questions was "mother's maiden name."
Since that's commonly used for verification by financial companies,
there was no way I was giving it to a system that would probably store
it in plaintext (easier to check against) and that anyway didn't store
particularly important information.

In that case I had five questions to choose from, and one of the
others was "favorite band", which falls under the ice cream problem.
More choices would have made me *much* less irritated by the site.
Now I wish I could remember what it was to warn others off....

Jennifer Berk

On Mon, 3 Mar 2008 16:48:13, Rob Tannen <rtannen at bresslergroup.com> wrote:
> Todd - Depending on the specifics of the question choices, it's not
> clear whether more questions is not in fact more efficient. One of
> the issues with "challenge questions" is that users may not have
> appropriate or memorable choices to select from. For example, your
> high school didn't have a mascot or your dad doesn't have a middle
> name.
>
> This is especially noticeable with an international user base who
> have less in common culturally to draw from. Therefore, more
> question choices increases the likelihood that there there are
> questions which users can readily come up with an answer for, rather
> than a limited choice where users struggle to find appropriate
> questions. In other words, more questions is more efficient from a
> mental task completion view, versus a speed of reading perspective.

3 Mar 2008 - 9:22pm
Todd Warfel
2003

On Mar 3, 2008, at 5:21 PM, Robert Barlow-Busch wrote:

> Yet so many people *passionately* prefer standard and wouldn't be
> caught
> dead driving an automatic.

And then there's the awkward irony of something like a corvette being
sold with an automatic transmission. There's just something about an
automatic sports car that's just ooo so wrong.

Cheers!

Todd Zaki Warfel
President, Design Researcher
Messagefirst | Designing Information. Beautifully.
----------------------------------
Contact Info
Voice: (215) 825-7423
Email: todd at messagefirst.com
AIM: twarfel at mac.com
Blog: http://toddwarfel.com
----------------------------------
In theory, theory and practice are the same.
In practice, they are not.

3 Mar 2008 - 9:23pm
Todd Warfel
2003

On Mar 3, 2008, at 5:26 PM, Elizabeth Buie wrote:

> No, the automatic would come out as the most learnable

Well, this is one very key factor in determining usability.

Cheers!

Todd Zaki Warfel
President, Design Researcher
Messagefirst | Designing Information. Beautifully.
----------------------------------
Contact Info
Voice: (215) 825-7423
Email: todd at messagefirst.com
AIM: twarfel at mac.com
Blog: http://toddwarfel.com
----------------------------------
In theory, theory and practice are the same.
In practice, they are not.

4 Mar 2008 - 9:30am
Elizabeth Buie
2004

Todd wrote:

> On Mar 3, 2008, at 5:26 PM, Elizabeth Buie wrote:
>> No, the automatic would come out as the most learnable

> Well, this is one very key factor in determining usability.

Yes, of course. But it does not constitute usability by its own self; there
are other factors.

Remember that usability does not exist in a vacuum; it depends on who is using
the product and in what context. I worked for a couple of years on an air
traffic control system, and "most learnable" in that context would definitely
NOT have been "most usable" because it would have slowed them down in operation
and could have made the skies much less safe.

Elizabeth

4 Mar 2008 - 9:35am
Elizabeth Buie
2004

Brandon Ward writes:

>I think everyone could agree that automatic is technically 'easier',

Depends on "by whom". It really *isn't* easier for me.

>Sorry if this isn't very coherent - but I think my point is in there
>somewhere.

I'm not so sure.

I drive a stick, never owned an automatic.
I use a Mac, never owned a Windows machine.

Go figure.

:-)

Elizabeth -- or maybe I'm just weird

4 Mar 2008 - 9:37am
Todd Warfel
2003

On Mar 4, 2008, at 9:30 AM, Elizabeth Buie wrote:

> Yes, of course. But it does not constitute usability by its own
> self; there are other factors.

That's my point. Many of the "usability professionals" I run into
don't see usability as multi-factor. Unfortunately, many of them
equate usability to efficiency and intuition, which are only a small,
albeit significant, part of the pie.

While intuition and efficiency are important parts, we can't forget
about learnability, satisfaction, and the ability to inevitably
complete a task/goal.

Cheers!

Todd Zaki Warfel
President, Design Researcher
Messagefirst | Designing Information. Beautifully.
----------------------------------
Contact Info
Voice: (215) 825-7423
Email: todd at messagefirst.com
AIM: twarfel at mac.com
Blog: http://toddwarfel.com
----------------------------------
In theory, theory and practice are the same.
In practice, they are not.

4 Mar 2008 - 9:48am
Elizabeth Buie
2004

Todd wrote:

> On Mar 4, 2008, at 9:30 AM, Elizabeth Buie wrote:

>> Yes, of course. But it does not constitute usability by its own self; there are other factors.

> That's my point. Many of the "usability professionals" I run into don't see usability as multi-factor.

I don't know how they can miss it, given how ISO 9241 defines usability. Or maybe
they are only self-described "usability professionals" and aren't very knowledgeable?
That would be highly unfortunate.

> Unfortunately, many of them equate usability to efficiency and intuition,

I can't say I know any who see it this way. But I do know many interaction
designers who think that the usability process is about nothing but evaluation. ;-)

> While intuition and efficiency are important parts, we can't forget about
> learnability, satisfaction, and the ability to inevitably complete a task/goal.

But wait a minute. In your first post on the subject, you said that the product
was less usable but yielded *greater* satisfaction, so I pointed out that your
comment ignored the satisfaction component of usability. Now you're telling *me*
that satisfaction is a component of usability?

What is wrong with this picture?

Elizabeth

4 Mar 2008 - 10:02am
Todd Warfel
2003

On Mar 4, 2008, at 9:48 AM, Elizabeth Buie wrote:

> I don't know how they can miss it, given how ISO 9241 defines
> usability.

Elizabeth, there are a lot of people out in the field, especially
marketing agencies, performing usability studies who haven't got a
clue that there's an ISO 9241 standard for it. You might be surprised
by this, but it's true.

> But wait a minute. In your first post on the subject, you said that
> the product was less usable but yielded *greater* satisfaction, so I
> pointed out that your comment ignored the satisfaction component of
> usability. Now you're telling *me* that satisfaction is a component
> of usability?
>
> What is wrong with this picture?

I include satisfaction in my definition of usability, but as I said
before, many HCI purists, especially in academia don't see it that
way. You might not see that in your work, but that might be due to the
industry you're focused in. Likewise, since I'm not involved in
government work, where we're not obligated to be bound by ISO
standards, I see it pretty often.

Cheers!

Todd Zaki Warfel
President, Design Researcher
Messagefirst | Designing Information. Beautifully.
----------------------------------
Contact Info
Voice: (215) 825-7423
Email: todd at messagefirst.com
AIM: twarfel at mac.com
Blog: http://toddwarfel.com
----------------------------------
In theory, theory and practice are the same.
In practice, they are not.

4 Mar 2008 - 10:13am
SemanticWill
2007

Elizabeth, there are a lot of people out in the field, especially
marketing agencies, performing usability studies who haven't got a
clue that there's an ISO 9241 standard for it. You might be surprised
by this, but it's true.
......

To the point:
"In the dusty institutions where usability standards gather to party with
each other, ISO 9241 is a bit of a celebrity. It is widely cited by people
who would be hard pushed to name any other standard, and* **parts of it are
virtually enshrined in law in some European countries. *But as is the fate
of many celebrities , *all most usability professionals know about the
standard is its name*."
- http://www.userfocus.co.uk/articles/ISO9241.html

>

4 Mar 2008 - 10:59am
Elizabeth Buie
2004

Todd writes:

>Elizabeth, there are a lot of people out in the field, especially
>marketing agencies,

Well, there you have it. They're marketing folks. IMNERHO they aren't usability
professionals; they're just calling themselves that.

>I include satisfaction in my definition of usability,

Then would you mind explaining what you had in mind when you said that a design
was less usable but more satisfying? Help me out here.

>You might not see that in your work, but that might be due to the
>industry you're focused in.

I have clients in the government, nonprofit, and commercial sectors.
Which industry did you have in mind? :-)

>Likewise, since I'm not involved in
>government work, where we're not obligated to be bound by ISO
>standards, I see it pretty often.

I am not talking about being "bound" by ISO standards. I am talking about
knowing the definition of usability, which happens to be best captured in
ISO 9241/11. This definition comes up rather frequently among people who
actually *are* usability professionals.

Elizabeth

4 Mar 2008 - 11:19am
Todd Warfel
2003

On Mar 4, 2008, at 10:59 AM, Elizabeth Buie wrote:

> Well, there you have it. They're marketing folks. IMNERHO they
> aren't usability
> professionals; they're just calling themselves that.

They call themselves that and they are doing usability research.
Companies are hiring them. I've seen the tapes. I've read the reports.
Whether we like it or not, and personally I don't really like it, the
reality is that these people are doing usability work and it's rather
common.

> Then would you mind explaining what you had in mind when you said
> that a design was less usable but more satisfying? Help me out here.

Can't. Satisfaction is included in my definition of usability. But as
I've said before, not everyone shares my definition of usability
(remember those marketing agencies doing usability work?).

> I have clients in the government, nonprofit, and commercial sectors.
> Which industry did you have in mind? :-)

Based on your site, it looks like your focus is on government and non-
profit. While we've done work for both, that's not our focus—we focus
more on commercial businesses. However, whenever we deal with
government/non-profit, we do have different "standards" we have to
abide by. For example, right now, we're doing work for a very large
bank. Actually, the company was acquired last year by a large bank.
So, small startup that's now part of one of the world's largest banks.
Last year, they weren't bound by 508 compliance and a few other
"standards" that this bank is. Now they are. So, now part of our work
is upgrading their systems for them.

And btw, it wasn't that they didn't care before, they weren't aware
and didn't have the budget for this type of work before. Now they are
aware and required so they have to create a budget for it.

> I am not talking about being "bound" by ISO standards. I am talking
> about knowing the definition of usability, which happens to be best
> captured in ISO 9241/11. This definition comes up rather frequently
> among people who actually *are* usability professionals.

Last year at UPA, I taught an entire day long tutorial on creating
data-driven design research persons. Part of that was how to use them
in the usability process. During our discussion of usability, not one
single person in the room of 50+ ever once referenced the ISO 9241
definition of usability. Now, while this isn't conclusive, I'd think
that a crowd at UPA is a pretty good representation of usability
professionals.

Additionally, I've spoken at a number of UPA groups and have not once
heard any reference to ISO 9241. I'm aware there's a standard, but
I'll bet you that most people I encounter in the field aren't as well
versed on it as you are and couldn't give me the ISO definition.

I'm not claiming that my experience is the entire total truth, clearly
it's not. But clearly, there's an entire universe out there doing
usability work who are totally and completely unaware of the ISO
standard definition.

Cheers!

Todd Zaki Warfel
President, Design Researcher
Messagefirst | Designing Information. Beautifully.
----------------------------------
Contact Info
Voice: (215) 825-7423
Email: todd at messagefirst.com
AIM: twarfel at mac.com
Blog: http://toddwarfel.com
----------------------------------
In theory, theory and practice are the same.
In practice, they are not.

4 Mar 2008 - 12:31pm
Patricia Garcia
2007

I'm a bit confused. You reference the fact that the form takes more
time and effort as meaning it's unusable. Efficiency (which is
what I assume you are referring to) doesn't just mean faster, it
means less wasted time. If the goal if the form was to be more
secure and adding more questions accomplished that goal, how is that
wasted time to look through more questions? Particularly when the
user satisfaction was high.

I don't know, but I would guess based on your results from testing
that this form is indeed usable.

As for the automatic/manual argument, both pass usablility depending
on your users goals. It's not a one size fits all solution.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=26755

5 Mar 2008 - 1:48pm
kiran mova
2007

what would be the usability implication if user is allowed to write their
own question and answer?

If open-ended is too vague, will it help to provide sample questions?

If sample questions are provided, will users tend to choose one of the
sample questions?

On Tue, 4 Mar 2008 09:31:44, Patricia Garcia <pgarcia413 at earthlink.net>
wrote:

> I'm a bit confused. You reference the fact that the form takes more
> time and effort as meaning it's unusable. Efficiency (which is
> what I assume you are referring to) doesn't just mean faster, it
> means less wasted time. If the goal if the form was to be more
> secure and adding more questions accomplished that goal, how is that
> wasted time to look through more questions? Particularly when the
> user satisfaction was high.
>
> I don't know, but I would guess based on your results from testing
> that this form is indeed usable.
>
> As for the automatic/manual argument, both pass usablility depending
> on your users goals. It's not a one size fits all solution.
>
>
> . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Posted from the new ixda.org
> http://www.ixda.org/discuss?post=26755
>
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help
>

6 Mar 2008 - 7:32am
Elizabeth Buie
2004

At 11:19 AM -0500 3/4/08, Todd Zaki Warfel wrote:
>On Mar 4, 2008, at 10:59 AM, Elizabeth Buie wrote:

>Whether we like it or not, and personally I don't really
>like it, the reality is that these people are doing usability work
>and it's rather common.

I agree. But that doesn't make them usability professionals.

>>Then would you mind explaining what you had in mind when you said
>>that a design was less usable but more satisfying? Help me out here.
>
>
>Can't. Satisfaction is included in my definition of usability.

Let me repeat. YOU said that a design was more satisfying but less usable.
How do you reconcile that discrepancy?

I am perfectly willing to agree that a design can be both more satisfying
and less usable, if the efficiency and/or effectiveness measures counter
the satisfaction measure strongly enough so that they bring down the overall
rating. But in your description of the "usability" of the design, the only
thing you mentioned in calling it "less usable" was its efficiency.
Therein lies my confusion.

>>I have clients in the government, nonprofit, and commercial
>>sectors. Which industry did you have in mind? :-)
>
>Based on your site, it looks like your focus is on government and
>non-profit.

Aha, you looked at my projects list. I need to update that. :-)

>Last year at UPA, I taught an entire day long tutorial on creating
>data-driven design research persons.

I would like to have taken that.

>not one single person in the room of 50+ ever once referenced the
>ISO 9241 definition of usability.

>Additionally, I've spoken at a number of UPA groups and have not
>once heard any reference to ISO 9241.

I didn't say that people talk about 9241, but that it's very common for
them to know that usability consists of effectiveness, efficiency, and
satisfaction. It's not necessary to mention those three factors by name,
to be guided by them. We often see people talk about "Can they do the
tasks?" "How long does it take them?" "How much do they like the product?"

Elizabeth
--
Elizabeth Buie
Principal Consultant
Luminanze Consulting, LLC
tel +1.301.943.4168
fax +1.301.949.9694
http://www.luminanze.com

6 Mar 2008 - 9:08am
Todd Warfel
2003

On Mar 6, 2008, at 7:32 AM, Elizabeth Buie wrote:

> I agree. But that doesn't make them usability professionals.

Semantics. Call them usability practitioners, call them "people
practicing usability," but they're getting paid for doing usability
work, which is one of the definitions of a professional.

> Let me repeat. YOU said that a design was more satisfying but less
> usable. How do you reconcile that discrepancy?
>
> I am perfectly willing to agree that a design can be both more
> satisfying and less usable, if the efficiency and/or effectiveness
> measures counter the satisfaction measure strongly enough so that
> they bring down the overall rating. But in your description of the
> "usability" of the design, the only thing you mentioned in calling
> it "less usable" was its efficiency. Therein lies my confusion.

First, satisfaction is only one factor in determining usability.
Second, participants were stalled a bit at the screen and took
significantly longer to fill out these three questions than what we've
seen in the past. Now, that's not rocket science since there were
close to 4x as many questions and they were each in random order.
Third, as I've already said and I'll say it again, many people doing
usability work focus on efficiency and think efficiency=usability. My
point is that it doesn't. Might be obvious to you, but speaking from
experience having watched and spoken to others doing usability work,
as well as having read usability reports from others doing usability
work, I can tell you, like it or not, that often the focus is mostly
on efficiency.

> I would like to have taken that.

Well, if you're at the IA summit, you'll be able to hear a talk about
it. It's not the workshop, but it will be a talk.

Cheers!

Todd Zaki Warfel
President, Design Researcher
Messagefirst | Designing Information. Beautifully.
----------------------------------
Contact Info
Voice: (215) 825-7423
Email: todd at messagefirst.com
AIM: twarfel at mac.com
Blog: http://toddwarfel.com
----------------------------------
In theory, theory and practice are the same.
In practice, they are not.

6 Mar 2008 - 9:12am
Todd Warfel
2003

On Mar 6, 2008, at 9:08 AM, Todd Zaki Warfel wrote:

> On Mar 6, 2008, at 7:32 AM, Elizabeth Buie wrote:
>
>> I agree. But that doesn't make them usability professionals.
>
> Semantics. Call them usability practitioners, call them "people
> practicing usability," but they're getting paid for doing usability
> work, which is one of the definitions of a professional.

BTW, I don't like it anymore than you that these people are doing
usability work. In fact, I'm rather bothered by it. But it is a
reality that it is happening and we need to be conscious of it. The
fact that you don't consider them professionals says something about
your confidence in their ability to do this work well, at least that
would be my interpretation. And with that I would agree.

Cheers!

Todd Zaki Warfel
President, Design Researcher
Messagefirst | Designing Information. Beautifully.
----------------------------------
Contact Info
Voice: (215) 825-7423
Email: todd at messagefirst.com
AIM: twarfel at mac.com
Blog: http://toddwarfel.com
----------------------------------
In theory, theory and practice are the same.
In practice, they are not.

6 Mar 2008 - 9:48am
Todd Roberts
2005

Maybe I'm interpreting this discussion wrong, but it seems to be belittling
the value of effectiveness and efficiency. Regardless of whether a product
(read: product or service) provides the user with some level of
satisfaction, it is better that it be effective and efficient than not. If a
company that has previously paid little or no attention to any of the stated
aspects of usability (and this seems to be a significant proportion of the
marketplace) brings in someone to improve efficiency and effectiveness, the
products will become more effective and efficient. This is almost certainly
a positive outcome.

Is it as good as if satisfaction was also addressed? Of course not. The more
factors impacting user experience that are addressed, the better the
product. Someone calling themselves a user experience professional could
easily bash a usability professional for "only" addressing effectiveness,
efficiency and satisfaction. This is where I find semantic arguments over
the definition of usability not particularly useful. What really matters is
which parts of the constellation of factors impacting user experience are
addressed, not which label is applied to some arbitrary subset of those
factors.

I do think these discussions highlight something important though, which is
the increasing recognition of the importance of a wide range of factors that
impact the quality of a product. This is leading to broadening definitions
of usability, IA, and IxD to be more encompassing of the whole experience.
If you ask any particular usability professional, IA or IxD what they do, my
guess is the answers would be very similar, whereas in the past that might
not have been the case.

-- A different Todd

> BTW, I don't like it anymore than you that these people are doing
> usability work. In fact, I'm rather bothered by it. But it is a
> reality that it is happening and we need to be conscious of it. The
> fact that you don't consider them professionals says something about
> your confidence in their ability to do this work well, at least that
> would be my interpretation. And with that I would agree.
>
>
> Cheers!
>
> Todd Zaki Warfel
>

6 Mar 2008 - 10:03am
Todd Warfel
2003

On Mar 6, 2008, at 9:48 AM, Todd Roberts wrote:

> I do think these discussions highlight something important though,
> which is the increasing recognition of the importance of a wide
> range of factors that impact the quality of a product.

I think this is the most important point of the discussion, or at
least my original intent.

Cheers!

Todd Zaki Warfel
President, Design Researcher
Messagefirst | Designing Information. Beautifully.
----------------------------------
Contact Info
Voice: (215) 825-7423
Email: todd at messagefirst.com
AIM: twarfel at mac.com
Blog: http://toddwarfel.com
----------------------------------
In theory, theory and practice are the same.
In practice, they are not.

Syndicate content Get the feed