Smart Filtering for Usability Requests

4 Apr 2006 - 4:38pm
8 years ago
9 replies
574 reads
fahdoo
2006

Hey,

This is a very interesting problem that I was asked to find a solution for
from my usability team lead.

Problem: We have a web form on our intranet which different teams can fill
out to request usability work. Currently the form is primarily textfields
asking for information about the project. The different project teams fill
these out without restriction.The team lead is receiving some forms that
have not much of a usability nature to them. Also, there is more work than
there are people to handle projects, so there is definately a need to
prioritize projects as they come in.

The proposed solution: some sort of a smart filtering system that can let
the team applying know if their project qualifies for usability testing as
soon as they click on the "submit button" (don't worry, we intend to let
them off nicely). I imagine we'd have to have a set of questions that could
determine the usability-ness of a project (given they enter the form
honestly); and we'd have to set some sort of cut-off value. Also, instead of
flat out rejecting the form, the Team Lead could be given a summary of where
the form is ranked on a "usability-ness scale" and let them make a decision
(i.e. its up to the Team Lead to reject based on the ranking, or to read the
form in more detail)

Does anyone have any suggestions, prior experience...on how to tackle this
sort of issue? There could be a wide range of projects that would want to
carry out a usability study, and so there needs to be a set of questions
with checkboxes/radio buttons that could be used in the smart algorithm.

Thanks

--
::Fahd Butt::
Systems Design Engineering - University of Waterloo
visualmetaphor.com
sofia-ai.org

Comments

4 Apr 2006 - 5:20pm
Todd Roberts
2005

My group is responsible for all things info-system-related, so we don't have
quite as much filtering to do as your group and are currently prioritizing
requests manually. As you mentioned, any algorithms you develop will be
gamed so there will need to be some other mechanism. How is your group
funded? Do you operate as an internal consultancy with chargebacks, or do
you have a set operating budget and have to take all comers? I ask because
this will make a difference in how you prioritize and your possibilities in
terms of staffing.

4 Apr 2006 - 9:38pm
fahdoo
2006

On 4/4/06, Todd Roberts <trrobert at gmail.com> wrote:
> How is your group
> funded? Do you operate as an internal consultancy with chargebacks, or do
> you have a set operating budget and have to take all comers?

Hey Todd

The chargebacks way is how our group works.

I don't think cost necessarily needs to be included into the filtering
system; the main concern is that there are some superfluous requests
coming in that aren't suitable for usability studies. This is probably
due to the fact that usability as such isn't quite well understood.

::Fahd Butt::
Systems Design Engineering - University of Waterloo
visualmetaphor.com
sofia-ai.org

5 Apr 2006 - 7:58am
Donna Maurer
2003

So the usability team want to push the burden of determining 'what is a usability
problem' back to the users to reduce the number of requests for usability work.

Some usability team!

Donna

On 4 Apr 2006 at 17:38, Fahd Butt wrote:

> [Please voluntarily trim replies to include only relevant quoted material.]
>
> Hey,
>
> This is a very interesting problem that I was asked to find a solution for
> from my usability team lead.
>
> Problem: We have a web form on our intranet which different teams can fill
> out to request usability work. Currently the form is primarily textfields
> asking for information about the project. The different project teams fill
> these out without restriction.The team lead is receiving some forms that
> have not much of a usability nature to them. Also, there is more work than
> there are people to handle projects, so there is definately a need to
> prioritize projects as they come in.
>
> The proposed solution: some sort of a smart filtering system that can let
> the team applying know if their project qualifies for usability testing as
> soon as they click on the "submit button" (don't worry, we intend to let
> them off nicely). I imagine we'd have to have a set of questions that could
> determine the usability-ness of a project (given they enter the form
> honestly); and we'd have to set some sort of cut-off value. Also, instead of
> flat out rejecting the form, the Team Lead could be given a summary of where
> the form is ranked on a "usability-ness scale" and let them make a decision
> (i.e. its up to the Team Lead to reject based on the ranking, or to read the
> form in more detail)
>
> Does anyone have any suggestions, prior experience...on how to tackle this
> sort of issue? There could be a wide range of projects that would want to
> carry out a usability study, and so there needs to be a set of questions
> with checkboxes/radio buttons that could be used in the smart algorithm.
>
> Thanks
>
> --
--
Donna Maurer
Maadmob Interaction Design

e: donna at maadmob.net
work: http://maadmob.com.au/
blog: http://maadmob.net/donna/blog/
AOL IM: maadmob

5 Apr 2006 - 8:31am
Josh Seiden
2003

This is an opportunity to grow and evangelize within
your organization. Instead of rejecting requests
automatically, you should try to:

1. be pleased that so many people want your services
2. provide education about what is/isn't usability
3. use this data to evangelize mgmt to provide greater
resources.
4. avoid automatic rejection at all costs!

In my consulting practice, my experience is that you
must talk to every lead personally, because *the* key
tactic in growing your business is relationship
building.

JS

> > Problem: We have a web form on our intranet which
> different teams can fill
> > out to request usability work.

> > Does anyone have any suggestions, prior
> experience...on how to tackle this
> > sort of issue?

5 Apr 2006 - 8:46am
Lisa Colvin
2005

Hi there-

I wholeheartedly agree with Josh on this one.

I'd like to point out though that in many organizations, we have to beg to
be _allowed_ to include testing in our project budgets. The fact that your
project teams are asking is terrific! Are there levels of testing you could
provide for certain sizes or types of projects? Sometimes we provide "office
hours" type assistance, where we take the time to do a quick review on
projects for which we aren't allowed to do full testing. Call it a sanity
check or whatever, but it's created some good bonds between departments.

Whatever you do, don't take it for granted that no matter how often people
are refused, they will still come back. The last thing you want is for
teams to think you don't care about or just won't be available for their
projects - or that you have an unusable usability process.

Hope that helps-

Lisa

-----Original Message-----
From: discuss-bounces at lists.interactiondesigners.com
[mailto:discuss-bounces at lists.interactiondesigners.com] On Behalf Of Josh
Seiden
Sent: Wednesday, April 05, 2006 6:32 AM
To: discuss at ixda.org
Subject: Re: [IxDA Discuss] Smart Filtering for Usability Requests

[Please voluntarily trim replies to include only relevant quoted material.]

This is an opportunity to grow and evangelize within
your organization. Instead of rejecting requests
automatically, you should try to:

1. be pleased that so many people want your services
2. provide education about what is/isn't usability
3. use this data to evangelize mgmt to provide greater
resources.
4. avoid automatic rejection at all costs!

In my consulting practice, my experience is that you
must talk to every lead personally, because *the* key
tactic in growing your business is relationship
building.

JS

> > Problem: We have a web form on our intranet which
> different teams can fill
> > out to request usability work.

> > Does anyone have any suggestions, prior
> experience...on how to tackle this
> > sort of issue?

________________________________________________________________
Welcome to the Interaction Design Association (IxDA)!
To post to this list ....... discuss at ixda.org
List Guidelines ............ http://listguide.ixda.org/
List Help .................. http://listhelp.ixda.org/
(Un)Subscription Options ... http://subscription-options.ixda.org/
Announcements List ......... http://subscribe-announce.ixda.org/
Questions .................. lists at ixda.org
Home ....................... http://ixda.org/
Resource Library ........... http://resources.ixda.org

5 Apr 2006 - 1:47pm
fahdoo
2006

Thanks for the replies,

Donna,

So the usability team want to push the burden of determining 'what is a
> usability
> problem' back to the users to reduce the number of requests for usability
> work.
>

We don't want them to define usability, but to define their project/needs
within some usability standard which would allow us to prioritize the
projects. It might not be the best way, but just one method of reducing
requests that aren't applicable from the point of entry. Any alternative
ideas that you may have?

Lisa,

Are there levels of testing you could
> provide for certain sizes or types of projects? Sometimes we provide
> "office
> hours" type assistance, where we take the time to do a quick review on
> projects for which we aren't allowed to do full testing. Call it a sanity
> check or whatever, but it's created some good bonds between departments.
>

That seems like a good idea. Our testing is usually small to mid-range (in
terms of participants).

Whatever you do, don't take it for granted that no matter how often people
> are refused, they will still come back. The last thing you want is for
> teams to think you don't care about or just won't be available for their
> projects - or that you have an unusable usability process.
>

Gotcha, I'll try to communicate this.

Josh,

This is an opportunity to grow and evangelize within
> your organization. Instead of rejecting requests
> automatically, you should try to:
>
> 1. be pleased that so many people want your services
> 2. provide education about what is/isn't usability
> 3. use this data to evangelize mgmt to provide greater
> resources.
> 4. avoid automatic rejection at all costs!
>

Yes, I wasn't really in favor of automatic rejection so I suppose educating
the teams about usability prior to anything else would be a good idea. I am
thinking of aiming the form to have more specific usability related
questions in order to allow our team to have a better understanding of the
other project teams needs. The form just might be too open-ended that the
pertinent information doesn't get through.

As an example, one of the projects that our usability team was commisioned,
ended up being transferred to the Beta group after a number of meetings with
the developers.

Yes, I do have my doubts that a form filtering system can prevent this in
the future, but thanks for all the input.

Thanks,
--
::Fahd Butt::
Systems Design Engineering - University of Waterloo
visualmetaphor.com
sofia-ai.org

5 Apr 2006 - 9:28am
Sarah Bloomer
2006

It's true that this is a nice problem to have--to be so much in demand
within your organization. I think it's interesting that you are getting
requests your team doesn't consider "usability".

Another approach is to decide what activities your usability team will
do, and what you can pass on to your projects teams. When I was with
The Hiser Group, we developed a toolkit (the Hiser Element) which was
designed for teams that had the problem you outline. The concept is that
instead of doing all the activities yourselves, you cross-train members
of the product team in user centered design. We then worked with new
usability teams on how position themselves: did they facilitate UCD
activities (such as contextual studies) and be the only ones allowed to
do usability testing? Did they own the UI design or just help product
teams with paper prototyping sessions and provide expertise in GUI
standards (a model we use at The MathWorks where we train analysts and
developers in paper prototyping).

I would like to understand what you mean by "usability-ness". Does it
have to do with the activities being requested? Or the value of the
project? Without going into too much detail, at the request of our
clients, we developed a way to assess projects for their impact on the
company (this may be more relevant to internal software development, not
commercial software companies). This was to help companies decide which
projects the usability team should take on themselves, and which they
should provide mentoring and facilitation, thereby enabling them to
support more projects in effective ways.

New models are being identified as well to address this issue of UCD
load. At last year's User Interface conference, Jared spoke about 3
models UIE has identified (Jared? Are you there?) which also address
this issue of how much UCD teams should take on themselves, and how we
can distribute UCD activities across project teams.

You can find out more about the toolkit here:
http://www.hiser.com.au/the_hiser_element_toolkit/the_hiser_element_tool
kit_-_overview.html

Sarah

+ + + + + + + + + + + + + + + + + + + + + + + + + + +
Sarah Bloomer, Senior Usability Specialist
The MathWorks
3 Apple Hill Drive
Natick, Massachusetts 01760
USA

e. sbloomer at mathworks.com
p. +508-647-7147
+ + + + + + + + + + + + + + + + + + + + + + + + + + +

-----Original Message-----
From: discuss-bounces at lists.interactiondesigners.com
[mailto:discuss-bounces at lists.interactiondesigners.com] On Behalf Of
Lisa Colvin
Sent: Wednesday, April 05, 2006 9:47 AM
To: discuss at ixda.org
Subject: Re: [IxDA Discuss] Smart Filtering for Usability Requests

[Please voluntarily trim replies to include only relevant quoted
material.]

Hi there-

I wholeheartedly agree with Josh on this one.

I'd like to point out though that in many organizations, we have to beg
to
be _allowed_ to include testing in our project budgets. The fact that
your
project teams are asking is terrific! Are there levels of testing you
could
provide for certain sizes or types of projects? Sometimes we provide
"office
hours" type assistance, where we take the time to do a quick review on
projects for which we aren't allowed to do full testing. Call it a
sanity
check or whatever, but it's created some good bonds between departments.

Whatever you do, don't take it for granted that no matter how often
people
are refused, they will still come back. The last thing you want is for
teams to think you don't care about or just won't be available for their
projects - or that you have an unusable usability process.

Hope that helps-

Lisa

-----Original Message-----
From: discuss-bounces at lists.interactiondesigners.com
[mailto:discuss-bounces at lists.interactiondesigners.com] On Behalf Of
Josh
Seiden
Sent: Wednesday, April 05, 2006 6:32 AM
To: discuss at ixda.org
Subject: Re: [IxDA Discuss] Smart Filtering for Usability Requests

[Please voluntarily trim replies to include only relevant quoted
material.]

This is an opportunity to grow and evangelize within
your organization. Instead of rejecting requests
automatically, you should try to:

1. be pleased that so many people want your services
2. provide education about what is/isn't usability
3. use this data to evangelize mgmt to provide greater
resources.
4. avoid automatic rejection at all costs!

In my consulting practice, my experience is that you
must talk to every lead personally, because *the* key
tactic in growing your business is relationship
building.

JS

> > Problem: We have a web form on our intranet which
> different teams can fill
> > out to request usability work.

> > Does anyone have any suggestions, prior
> experience...on how to tackle this
> > sort of issue?

________________________________________________________________
Welcome to the Interaction Design Association (IxDA)!
To post to this list ....... discuss at ixda.org
List Guidelines ............ http://listguide.ixda.org/
List Help .................. http://listhelp.ixda.org/
(Un)Subscription Options ... http://subscription-options.ixda.org/
Announcements List ......... http://subscribe-announce.ixda.org/
Questions .................. lists at ixda.org
Home ....................... http://ixda.org/
Resource Library ........... http://resources.ixda.org

________________________________________________________________
Welcome to the Interaction Design Association (IxDA)!
To post to this list ....... discuss at ixda.org
List Guidelines ............ http://listguide.ixda.org/
List Help .................. http://listhelp.ixda.org/
(Un)Subscription Options ... http://subscription-options.ixda.org/
Announcements List ......... http://subscribe-announce.ixda.org/
Questions .................. lists at ixda.org
Home ....................... http://ixda.org/
Resource Library ........... http://resources.ixda.org

5 Apr 2006 - 4:08pm
Lada Gorlenko
2004

SB> Another approach is to decide what activities your usability team will
SB> do, and what you can pass on to your projects teams.

This is very much in line with what Jared Spool describes as the three
models of design team organisation: 1) Consult, 2) Review & Approve,
and 3) Educate & Administrate.

Highly recommend. For details, see
http://www.uie.com/brainsparks/2006/03/28/ia-summit-presentation-we-are-not-alone/

Lada

5 Apr 2006 - 4:26pm
fahdoo
2006

Hey Sarah,

Thanks for the really interesting reply. I checked out the link you sent and
it seems to be in line with what I'm looking for.
At the place I work at right now, the usability group is still reasonably
young and even unknown to some groups. There definately is a push by our
team to make usability a part of the entire product development process,
instead of as an afterthought. As usability becomes part of the design
philosophy of each team, I'm sure this "problem" won't be much of an issue.

On 4/5/06, Sarah Bloomer <Sarah.Bloomer at mathworks.com> wrote:
>
> I would like to understand what you mean by "usability-ness". Does it
> have to do with the activities being requested? Or the value of the
> project?
>

By usability-ness I was referring to the activities they are requesting.
There are some activities that our Beta team would be more equipped to carry
out. Product testing is different from what Usability is for.

Thanks

--
::Fahd Butt::
Systems Design Engineering - University of Waterloo
visualmetaphor.com
sofia-ai.org

Syndicate content Get the feed