How much setup information to give test participants
10 Nov 2008 - 1:36pm
5 years ago
I'm preparing a usability test on a major redesign of an online
marketplace for manufacturing. Our customers are going to see major
changes to the way they source RFQs using our online app. I am planning
to go into the interviews without telling the participants what has
changed. There will be emails and online training before and after the
launch. However, we all know people usually skip over this material.
Would it be fair to give the participants a brief explanation about how
the process has changed? I am inclined NOT to do this for three reasons:
1) Don't want to lead the use
2) Want to determine knowledge gaps so we can adjust launch
communications and online support
3) There will be limited help text and help icons (for popup help). We
will see in the test if the users read or click help and if they
contact customer service (email or chat). This will tell us where we
need to beef up the support.
On one hand we want the test to help determine where gaps are so we can
fix issues and develop support materials. But in a test setting, they
will read whatever instructional content we give them for each task and
we can't assume our customers will actually do this when we launch.