Should an e-commerce design agency test the usability of its own designs?

9 Nov 2009 - 2:35pm
4 years ago
4 replies
870 reads
Paul Bryan
2008

Recently I was on an e-commerce strategy project. I received a
usability test report that the previous agency had produced after
testing their own design work. I went back to the source tapes and
there seemed to be a dramatic difference between the level of
problems users were having in the sessions, and the resulting report.

I know it's convenient for e-commerce site owners to get an
integrated package, esp. when large MSA's are in place. And trying
to keep ahead of Agile cycles puts strain on the schedule and number
of partners. But I'm just wondering if readers of this list feel
like there is an inherent conflict of interest, or if testing is
viewed as a normal component of a design partner relationship.

Paul Bryan
Usography (http://www.usography.com)
Blog: Virtual Floorspace (http://www.virtualfloorspace.com)
Linked In: http://www.linkedin.com/in/uxexperts

Comments

9 Nov 2009 - 4:12pm
Kristen Connor
2009

Paul, I think your example speaks more to a lack of professionalism on
the part of the previous agency than an inherent conflict of interest.

At my firm, Lofthouse Design, we have always conducted usability
testing on the sites and applications we design. In doing so, we find
opportunities to improve the product we deliver to our clients -- and
we do not take any criticism as an attack on our design skills.

For example, we recently tested an online enrollment application with
a sample of Medicare-age seniors. Although no users complained, we
observed first-hand that users struggled to use standard form
controls like radio buttons. They spent unnecessary time and effort
trying to click on a small target.

Based on this observation, we identified an opportunity to improve
the usability of basic form controls. We created a set of custom UI
elements that visually highlight whenever the user mouses over any
part of the radio button or its label, creating a much bigger target
for the user to click.

Had we not conducted the user testing ourselves, we may not have seen
such a subtle usability problem -- nor could we have come up with an
innovative design solution as we did in this case.

Hope this example helps.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=47287

10 Nov 2009 - 9:15am
Mark Schraad
2006

I think there is great benefit to having an independent (person or
group) do the testing of a design. I also think there is benefit to
having product managers generate the use cases designers work to
solve for. Additionally, having the back end development team do the
QA is troublesome.

Some level of self fulfilling prophesy is likely to find its way in.
Isn't that why we go to someone else for things as simple as prof
reading? What I don't think, is that it is necessarily cause for
inditement. They may very well have had separate staff do the
testing. If you understand how agencies work, then you know that it
is very much about billables... and with that you must have
deliverables. It is very rare for an agency to recommend another
group in that situation.

Mark

On Nov 9, 2009, at 11:35 AM, Paul Bryan wrote:

> Recently I was on an e-commerce strategy project. I received a
> usability test report that the previous agency had produced after
> testing their own design work. I went back to the source tapes and
> there seemed to be a dramatic difference between the level of
> problems users were having in the sessions, and the resulting report.
>
>
> I know it's convenient for e-commerce site owners to get an
> integrated package, esp. when large MSA's are in place. And trying
> to keep ahead of Agile cycles puts strain on the schedule and number
> of partners. But I'm just wondering if readers of this list feel
> like there is an inherent conflict of interest, or if testing is
> viewed as a normal component of a design partner relationship.
>
> Paul Bryan
> Usography (http://www.usography.com)
> Blog: Virtual Floorspace (http://www.virtualfloorspace.com)
> Linked In: http://www.linkedin.com/in/uxexperts
>
>
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help

10 Nov 2009 - 4:17pm
Jesse Zolna
2008

I agree that this sounds like a lack of professionalism. Maybe they
did not spot the same issues that you saw? If they saw the issues, I
am not sure why they would not have taken those learnings to heart and
used them to improve the design. In the end, better design would make
them look better than some report %u201Cvalidating%u201D the usability
of their design.

Better design (however you define it) should be the ultimate goal of
every project, and every and project within the project should
support that goal. Right? I don't know why else they would have
done usability testing, except to bill more hours, I suppose.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=47287

10 Nov 2009 - 4:44pm
Alan James Salmoni
2008

It sounds a little like an own goal here, but only work done by
someone else would show this up. If their usability testing doesn't
show up defects in their design, then I would be asking what are they
being paid for? (except for being able to tick the relevant box)

I guess it's one of those difficult questions for companies to ask
themselves when considering employing someone: if they are expert
enough in a field, why employ someone else? And if not, how do they
know who are the real experts?

This explains why a list of previous clients can be the best selling
point though I've known of products and services that have gained a
good client list based upon the strength of their client list ("they
must be good, look at the big names that use them!").

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=47287

Syndicate content Get the feed