My client contact, several of my contacts employees, and the founder of the
company have indicated a strong belief that, based on user surveys they have
conducted, their site is "pretty darn good".
Part of the process I am working to implement will include user testing during
the design phase. Does anyone have good counter points to why user surveys
should be taken with a grain of salt and not considered the
"end-all-be-all" judgment of a website.
Note: I haven't yet had the chance to review the surveys so I can't comment
as to the quality of the questions (i.e. are they leading). I do know that they
conducted the surveys by emailing users and asking them to participate.
Examples I thought of:
When you obtain survey
results based on an existing design that is 'in the wild' users are
typically compelled to be nice. People taking the survey don't want to be
If you test a prototype with users who understand that, what they are
seeing, is not final you are more likely to get honest feedback.
Typically users who volunteer
to take the time to complete a survey are happy with the product in
question. People who are unhappy with a product/site complain to
customer support and are annoyed already and therefore won't waste any
more time to complete a survey. We need to hear the bad stuff as
well as the good.