Usability testing a build and supplementing it with paper mocks

9 Jul 2008 - 8:38am
6 years ago
3 replies
426 reads
achong
2006

I'm currently in the process of usability testing a build. The build
consists of a lot of functionality but some of those parts are not
connected to the main navigation. People can still access the
functionality but not from the correct places.

I'm considering providing the user tasks to complete and filling in
the missing flow pieces with paper mocks. Do you think it's going to
be too disorientating going from a working build to a static mock or
will people generally be able to make that leap of faith and
understand they are still moving within the flow of the site?

Alternatively, I could test the interaction design for the functioning
pieces exclusively but I feel like there would be a missed opportunity
to test the IA/nomenclature and a wholistic view of the overall
concept of the site.

Hope that made sense. Thanks for any feedback

--
Adrian Chong
www.adrianchong.com/blog

Comments

9 Jul 2008 - 8:45am
Todd Warfel
2003

On Jul 9, 2008, at 10:38 AM, Adrian Chong wrote:

> I'm considering providing the user tasks to complete and filling in
> the missing flow pieces with paper mocks. Do you think it's going to
> be too disorientating going from a working build to a static mock or
> will people generally be able to make that leap of faith and
> understand they are still moving within the flow of the site?
>
> Alternatively, I could test the interaction design for the
> functioning pieces exclusively but I feel like there would be a
> missed opportunity to test the IA/nomenclature and a wholistic view
> of the overall
> concept of the site.

I'd stay away from paper mocks and instead put the digital images of
what you're going to print out into a PowerPoint or HTML pages w/image
maps the participant can click through. This still keeps things "on-
line" and is less disruptive.

Additionally, one thing we've found out recently through some A/B
testing is that if there is anything where color might impact the
usability/predictability/legibility of the product, then the color and
contrast between printed pieces and on screen actually biases the
results. We found this out first hand — same screens, but printed
versions compared to on screen versions gave us almost completely
different results.

Cheers!

Todd Zaki Warfel
President, Design Researcher
Messagefirst | Designing Information. Beautifully.
----------------------------------
Contact Info
Voice: (215) 825-7423
Email: todd at messagefirst.com
AIM: twarfel at mac.com
Blog: http://toddwarfel.com
Twitter: zakiwarfel
----------------------------------
In theory, theory and practice are the same.
In practice, they are not.

9 Jul 2008 - 2:07pm
Pawson, Mark
2007

I agree with Todd. We had to do this for a test and had screen
captures of the functionality that was not tied in to the main
functionality sitting on the task bar. I then acted the part of the
computer and launched these screen captures when the participant
choose the correct control or on prompting said they would do X.
It worked quite well. However, that being said the fully functioning
prototype gave us superior feedback because it allowed us to sit back
and observe the participant playing.

A similar thread to peruse http://www.ixda.org/discuss.php?post=23254

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=31162

10 Jul 2008 - 6:04am
Chauncey Wilson
2007

Like Mark,

I've done this before and switched between paper and funtioning prototype
and even an animated PowerPoint which we revealed when we got to the point
of a feature that wasn't implemented. I've also used the mix-fidelity
approach relatively late in development when a major requirement was
suddenly thrust on the product team. I found it quite useful to do a few
practice runs internally to get the transition down and work on the
instructions in the facilitator's script. If you are doing something
remotely, you might consider using a PowerPoint (or another interactive
tool) as a fill-in for the particular missing functionality.

There was a great paper by Desiree Sy, one of my colleagues at Autodesk
(then of Alias) where video and paper prototypes were mixed with working
features in a multi-modal prototype test. I'll see if I can get hold of a
copy of the paper.

Chauncey

On Wed, Jul 9, 2008 at 4:07 PM, Mark Pawson <mark.pawson at geomodeling.com>
wrote:

> I agree with Todd. We had to do this for a test and had screen
> captures of the functionality that was not tied in to the main
> functionality sitting on the task bar. I then acted the part of the
> computer and launched these screen captures when the participant
> choose the correct control or on prompting said they would do X.
> It worked quite well. However, that being said the fully functioning
> prototype gave us superior feedback because it allowed us to sit back
> and observe the participant playing.
>
> A similar thread to peruse http://www.ixda.org/discuss.php?post=23254
>
>
> . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Posted from the new ixda.org
> http://www.ixda.org/discuss?post=31162
>
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help
>

Syndicate content Get the feed