Indexing / storing usability study results

5 Jan 2010 - 12:17pm
4 years ago
3 replies
611 reads
Audrey Crane
2009

I am working in an organization that is focused on doing quick,
lightweight usability research, especially once the work has a good
foundation: bringing a design into a coffee shop, for example,
running 5 quick (15 minute) studies, making changes, showing 3 more
people the next day, etc.

In this model, I've seen things get revised and re-tested 10 or more
times.

At the same time, there is a formal research organization that would
like to have all the results organized and indexed so they can find
them later. Ideally they'd like to have a standard report completed
for every round, but 10 reports is somewhat onerous.

What the researchers are currently doing for the team is posting
video with 3-5 key takeaways from every session on basecamp.

Does anyone have any experience with or suggestions for a solution
that:
- doesn't burden what's supposed to be a very lightweight /
flexible testing process
- makes materials available an at least a minimally organized fashion
so people later can search for things like "filter testing", or
whatever
- keeps people not immediately involved appraised of progress

Comments

5 Jan 2010 - 8:41pm
Nickgould
2009

This is a GREAT question and something we have struggled with, too...
We don't have the answer yet, but one interesting tool we've
discovered is the indexing function in Acrobat. I don't have the
details at hand at the moment, but basically you can index multiple
PDF files and make them easily searchable. Keywords are located and
highlighted on a per-document level. If you index your summaries
this way, it would be very quick and easy to find a reference to a
particular keyword. Of course, knowing that this indexing will happen
informs the way the summaries are written to insure that appropriate
keywords are present. The next challenge is to make sure that
everyone uses the same terminology to refer to the same issue /
item...

Hope this is helpful!

Nick

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=48206

5 Jan 2010 - 9:03pm
Chauncey Wilson
2007

I've long been of the opinion that part of our UX infrastructure
should be a usability/UX/design database where we can track the
results of all types of user research/design critiques, field data,
and other types of data. The data can then be cataloged in many ways
and allow UX and others to:

1. Look at common problems across different products as well as the
same problem within a product. This would be useful for consistency
(of many different types) and the development of patterns, standards,
and guidelines.
2. Track the work of the team and metrics like the number of users,
effort spent on testing.
3. Conduct meta-analyses across different products.
4. Track what gets fixed.
and much more.

With a database, you could tag problems in many ways and also tag user
requirements (since one of the biggest problems in product design is
obtaining good requirements).

I've worked up an Access database as a pilot for a usability database
and loaded it with over 5000 items from open-ended surveys, usability
tests, user feedback, data from interviews, etc. If you set up your
templates for recording data for your studies in something like Excel
and use the same column headings, you can import into Access (as an
example) and merge the results. With each set of data, you would
indicate the type of study, the date, the method used, etc. You could
tag something as an "idea", a "requirement", a "problem", or some
other general tags and look across studies for patterns of different
sorts.

You could build a form as a front end to the database and allow others
to access some aspects of the database. You could link to the reports
themselves, but the database would allow you to slice and dice your
data and do some powerful triangulation that would make your team very
powerful and persuasive. I think that one of the flaws of our field
is that we often gather immense amount of data, but have no easy way
to look at it from different perspectives easily. Wikis allow some
tagging, but I don't think that is sufficient from the kind of
analyses that would yield significant benefits.

If you develop a database, you will need to plan some education about
what meta data are important and how to enter data consistently.

Chauncey

On Tue, Jan 5, 2010 at 4:17 AM, Audrey Crane <audcrane at gmail.com> wrote:
> I am working in an organization that is focused on doing quick,
> lightweight usability research, especially once the work has a good
> foundation: bringing a design into a coffee shop, for example,
> running 5 quick (15 minute) studies, making changes, showing 3 more
> people the next day, etc.
>
> In this model, I've seen things get revised and re-tested 10 or more
> times.
>
> At the same time, there is a formal research organization that would
> like to have all the results organized and indexed so they can find
> them later. Ideally they'd like to have a standard report completed
> for every round, but 10 reports is somewhat onerous.
>
> What the researchers are currently doing for the team is posting
> video with 3-5 key takeaways from every session on basecamp.
>
> Does anyone have any experience with or suggestions for a solution
> that:
> - doesn't burden what's supposed to be a very lightweight /
> flexible testing process
> - makes materials available an at least a minimally organized fashion
> so people later can search for things like "filter testing", or
> whatever
> - keeps people not immediately involved appraised of progress
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help
>

3 Feb 2010 - 12:53pm
Audrey Crane
2009

We are playing with the Acrobat indexing function as that seems like a
good candidate for us -- it doesn't require anyone to write in a
specific format or really do anything special, but it does make it
seem vaguely possible that you could find a needle in a haystack.

Thanks!

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=48206

Syndicate content Get the feed