Should an e-commerce design agency test the usability of its own designs?

Paul, I think your example speaks more to a lack of professionalism on the part of the previous agency than an inherent conflict of interest.

At my firm, Lofthouse Design, we have always conducted usability testing on the sites and applications we design. In doing so, we find opportunities to improve the product we deliver to our clients — and we do not take any criticism as an attack on our design skills.

For example, we recently tested an online enrollment application with a sample of Medicare-age seniors. Although no users complained, we observed first-hand that users struggled to use standard form controls like radio buttons. They spent unnecessary time and effort trying to click on a small target.

Based on this observation, we identified an opportunity to improve the usability of basic form controls. We created a set of custom UI elements that visually highlight whenever the user mouses over any part of the radio button or its label, creating a much bigger target for the user to click.

Had we not conducted the user testing ourselves, we may not have seen such a subtle usability problem — nor could we have come up with an innovative design solution as we did in this case.

Hope this example helps.

Posted from the new ixda.org
http://www.ixda.org/discuss?post=47287

I think there is great benefit to having an independent (person or group) do the testing of a design. I also think there is benefit to having product managers generate the use cases designers work to solve for. Additionally, having the back end development team do the QA is troublesome.

Some level of self fulfilling prophesy is likely to find its way in. Isn't that why we go to someone else for things as simple as prof reading? What I don't think, is that it is necessarily cause for inditement. They may very well have had separate staff do the testing. If you understand how agencies work, then you know that it is very much about billables... and with that you must have deliverables. It is very rare for an agency to recommend another group in that situation.

Mark

On Nov 9, 2009, at 11:35 AM, Paul Bryan wrote:

Recently I was on an e-commerce strategy project. I received a usability test report that the previous agency had produced after testing their own design work. I went back to the source tapes and there seemed to be a dramatic difference between the level of problems users were having in the sessions, and the resulting report. I know it's convenient for e-commerce site owners to get an integrated package, esp. when large MSA's are in place. And trying to keep ahead of Agile cycles puts strain on the schedule and number of partners. But I'm just wondering if readers [trim]

I agree that this sounds like a lack of professionalism. Maybe they did not spot the same issues that you saw? If they saw the issues, I am not sure why they would not have taken those learnings to heart and used them to improve the design. In the end, better design would make them look better than some report %u201Cvalidating%u201D the usability of their design.

Better design (however you define it) should be the ultimate goal of every project, and every and project within the project should support that goal. Right? I don't know why else they would have done usability testing, except to bill more hours, I suppose.

Posted from the new ixda.org
http://www.ixda.org/discuss?post=47287

It sounds a little like an own goal here, but only work done by someone else would show this up. If their usability testing doesn't show up defects in their design, then I would be asking what are they being paid for? (except for being able to tick the relevant box)

I guess it's one of those difficult questions for companies to ask themselves when considering employing someone: if they are expert enough in a field, why employ someone else? And if not, how do they know who are the real experts?

This explains why a list of previous clients can be the best selling point though I've known of products and services that have gained a good client list based upon the strength of their client list ("they must be good, look at the big names that use them!" ).

Posted from the new ixda.org
http://www.ixda.org/discuss?post=47287