exploratory protocol focus group

21 Jun 2005 - 1:01pm
9 years ago
4 replies
734 reads
Wendy Fischer
2004

What is an exploratory protocol focus group?

I am assuming that it is a focus group. However, I am wondering what the benefits/disadvantages are of going with an exploratory protocol focus group as opposed to just doing a usability test (say this relates to testing a cell phone).

I need some ammo on why we should focus on doing usability tests as opposed to doing "exploratory protocol focus groups.." To me testing will allow us to get better feedback on both a qualitative and quantitative basis. Particularly since we are focusing on things at a finer detail level as it pertains to a transaction, I think a usability test is a better protocol than a focus group.

-Wendy

Comments

21 Jun 2005 - 2:26pm
Ted Boren
2005

When you say "exploratory," I assume you mean this is quite early in the design
& definition process. If the goal is to get audience & task insights very early
in the development process, then a usability test may not get you what you need
either, especially if it's a lab test. A field test of the existing version of a
product may be better, or perhaps a contextual inquiry or other field method. A
focus group per se is iffy unless it can be very firmly grounded in the
participants' reality--by asking for specific examples, walking retrospectively
through actual breakdowns, etc. But even so, you're likely to get a highly
watered down account without the detail you need to improve the design. Better
to watch & interview people in context when possible.

Good luck,
Ted Boren

>>> "Wendy Fischer" <erpdesigner at yahoo.com> 6/21/2005 12:01:41 PM >>>
What is an exploratory protocol focus group?

I am assuming that it is a focus group. However, I am wondering what the
benefits/disadvantages are of going with an exploratory protocol focus group as
opposed to just doing a usability test (say this relates to testing a cell
phone).

I need some ammo on why we should focus on doing usability tests as opposed to
doing "exploratory protocol focus groups.." To me testing will allow us to get
better feedback on both a qualitative and quantitative basis. Particularly since
we are focusing on things at a finer detail level as it pertains to a
transaction, I think a usability test is a better protocol than a focus group.

-Wendy

------------------------------------------------------------------------------
This message may contain confidential information, and is
intended only for the use of the individual(s) to whom it
is addressed.
------------------------------------------------------------------------------

21 Jun 2005 - 2:34pm
Wendy Fischer
2004

My thoughts exactly. I would think that this would be something that occurs early in the design phase. However, somebody is offering it as an alternative to usability testing on a product that is in production. My plan was to focus on usability testing as a way of catching user errors in the production piece and focus on specific areas at a more granular level so we could increase the conversion rate. I don't feel that a focus group is the right venue to find usability problems within an interface.

Ted Boren <BorenMT at ldschurch.org> wrote:
When you say "exploratory," I assume you mean this is quite early in the design & definition process. If the goal is to get audience & task insights very early in the development process, then a usability test may not get you what you need either, especially if it's a lab test. A field test of the existing version of a product may be better, or perhaps a contextual inquiry or other field method. A focus group per se is iffy unless it can be very firmly grounded in the participants' reality--by asking for specific examples, walking retrospectively through actual breakdowns, etc. But even so, you're likely to get a highly watered down account without the detail you need to improve the design. Better to watch & interview people in context when possible.

Good luck,
Ted Boren

>>> "Wendy Fischer" <erpdesigner at yahoo.com> 6/21/2005 12:01:41 PM >>>
What is an exploratory protocol focus group?

I am assuming that it is a focus group. However, I am wondering what the benefits/disadvantages are of going with an exploratory protocol focus group as opposed to just doing a usability test (say this relates to testing a cell phone).

I need some ammo on why we should focus on doing usability tests as opposed to doing "exploratory protocol focus groups.." To me testing will allow us to get better feedback on both a qualitative and quantitative basis. Particularly since we are focusing on things at a finer detail level as it pertains to a transaction, I think a usability test is a better protocol than a focus group.

-Wendy

------------------------------------------------------------------------------
This message may contain confidential information, and is
intended only for the use of the individual(s) to whom it
is addressed.
------------------------------------------------------------------------------

21 Jun 2005 - 3:39pm
Josh Galban
2005

I guess I’ll shed my lurker’s cloak to respond to this.

I would probably position it in these terms (depending on the personalities
involved): focus groups are attitudinal in nature. They attempt to uncover
general impressions and biases. Usability tests are process-oriented. They
delineate/validate the sequence of steps involved in a process and identify
the specific points of failure (and success).

I have frequently used exploratory usability tests with a think-aloud protocol
early in the design process to get early feedback on a prototype/simulation.
Information produced through this process tends to be of two types: (1) missed
requirements, and (2) specific interaction design refinements that improve the
implementation of existing requirements. I also glean some prioritization of
requirements.

I’m guessing that the folks proposing an “exploratory protocol focus
group” would like to test out new product concepts / features for group
feedback. There’s a ton of psychological research that speaks to the
inability of individuals to (1) predict what they will like/dislike, (2)
accurately report on an experience while they are having it, or (3) recall how
they felt about an experience while they are having it.

“The Paradox of Choice” by Barry Schwartz summarizes data from some of
that research: http://tinyurl.com/bkeep

Overall, I think the way you frame the discussion will depend upon the people
you work with, their familiarity with the available methods and their
religious biases.

--Josh

------ Original Message ------
Received: Tue, 21 Jun 2005 11:01:48 AM PDT
From: Wendy Fischer <erpdesigner at yahoo.com>
To: discuss at ixdg.orgCc:
Subject: [ID Discuss] exploratory protocol focus group

[Please voluntarily trim replies to include only relevant quoted material.]

What is an exploratory protocol focus group?

I am assuming that it is a focus group. However, I am wondering what the
benefits/disadvantages are of going with an exploratory protocol focus group
as opposed to just doing a usability test (say this relates to testing a cell
phone).

I need some ammo on why we should focus on doing usability tests as opposed to
doing "exploratory protocol focus groups.." To me testing will allow us to
get better feedback on both a qualitative and quantitative basis.
Particularly since we are focusing on things at a finer detail level as it
pertains to a transaction, I think a usability test is a better protocol than
a focus group.

-Wendy

_______________________________________________
Welcome to the Interaction Design Group!
To post to this list ....... discuss at ixdg.org
(Un)Subscription Options ... http://discuss.ixdg.org/
Announcements List ......... http://subscribe-announce.ixdg.org/
Questions .................. lists at ixdg.org
Home ....................... http://ixdg.org/

21 Jun 2005 - 3:57pm
hlysnan
2005

It seems perhaps that the notion of exploratory user
testing, as in a cognitive walkthrough of a design in
its preliminary stages, is being coopted from a
one-on-one interview to a one-on-many interview.
Normally these types of interview techniques are, as
many have already noted, deployed at the begining of
the design phase, not after the production and QA test
cycle has begun. Most of these approaches fall into
the formative stage of testing when collecting
qualitative information from users is most helpful.

At some point, however, which is where you seem to be
in the design cycle, there comes a point to meausure
and trap quantitative results that reflect just how
good (or not so good, heaven forbid) the design is and
what to do about it. This is generally refered to as
summative testing.

By what has been shared so far, it is difficult to say
what the provider has in store when they refer to an
Exploratory Focus Group, but it would appear
challeging to validate a certain trend or trends in
user acceptance, behavior, error, or error recovery of
the working design. If, indeed, the exploratory focus
group is intended to trap more qualitative feedback
form users, how are you to prioirtize what to fix
first, second, third,... before the launch? A few
facts might help....

hlysnan

One posible

--- Wendy Fischer <erpdesigner at yahoo.com> wrote:

> [Please voluntarily trim replies to include only
> relevant quoted material.]
>
> What is an exploratory protocol focus group?
>
> I am assuming that it is a focus group. However, I
> am wondering what the benefits/disadvantages are of
> going with an exploratory protocol focus group as
> opposed to just doing a usability test (say this
> relates to testing a cell phone).
>
> I need some ammo on why we should focus on doing
> usability tests as opposed to doing "exploratory
> protocol focus groups.." To me testing will allow
> us to get better feedback on both a qualitative and
> quantitative basis. Particularly since we are
> focusing on things at a finer detail level as it
> pertains to a transaction, I think a usability test
> is a better protocol than a focus group.
>
> -Wendy
>
>
>
> _______________________________________________
> Welcome to the Interaction Design Group!
> To post to this list ....... discuss at ixdg.org
> (Un)Subscription Options ...
> http://discuss.ixdg.org/
> Announcements List .........
> http://subscribe-announce.ixdg.org/
> Questions .................. lists at ixdg.org
> Home ....................... http://ixdg.org/
>

Hlysnan

____________________________________________________
Yahoo! Sports
Rekindle the Rivalries. Sign up for Fantasy Football
http://football.fantasysports.yahoo.com

Syndicate content Get the feed