analyzing qualitative data
14 Nov 2007 - 10:41pm
10 replies
632 reads
oliver green
2006
Hello all,
I have a background in statistics and it is easy for me to analyze
quantitative data, but I am at a loss when it comes to analyzing
qualitative data. I bought the book Qualitative Data Analysis : An
Expanded Sourcebook by Alan Michael Huberman, but it was difficult to
follow since it did not give examples that I can relate to. I would
really appreciate it if someone could point me to information that
will help me learn more about this topic.
Thanks,
Oliver
Get the feed
Comments
Oliver,
Do you have specific examples of the type of data you'll be analyzing? It
will help people offer targeted suggestions.
Steve
On 15/11/2007, oliver green <oliverhci at gmail.com> wrote:
>
> Hello all,
>
> I have a background in statistics and it is easy for me to analyze
> quantitative data, but I am at a loss when it comes to analyzing
> qualitative data. I bought the book Qualitative Data Analysis : An
> Expanded Sourcebook by Alan Michael Huberman, but it was difficult to
> follow since it did not give examples that I can relate to. I would
> really appreciate it if someone could point me to information that
> will help me learn more about this topic.
>
> Thanks,
> Oliver
> ________________________________________________________________
> *Come to IxDA Interaction08 | Savannah*
> February 8-10, 2008 in Savannah, GA, USA
> Register today: http://interaction08.ixda.org/
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help
>
--
----------------------------------------------
Steve 'Doc' Baty B.Sc (Maths), M.EC, MBA
Director, User Experience Strategy
Red Square
P: +612 8289 4930
M: +61 417 061 292
Member, UPA - www.upassoc.org
Member, IxDA - www.ixda.org
Member, Web Standards Group - www.webstandardsgroup.org
Good catch. I am analyzing notebook diaries recording thoughts and
observations during the usage of the system and audio interviews
conducted at the end of the session.
Thanks,
Oliver
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=22609
What data do you want to analyze (Qualitative).
There are two schools of thought, one the purists, who do not like doing
things like t tests with Likert scale ratings, and another school of
thought, where people do such tests.
What exactly is the data you have and what is it that you want to analyze?
- Kunal.
On 11/15/07, oliver green <oliverhci at gmail.com> wrote:
>
> Hello all,
>
> I have a background in statistics and it is easy for me to analyze
> quantitative data, but I am at a loss when it comes to analyzing
> qualitative data. I bought the book Qualitative Data Analysis : An
> Expanded Sourcebook by Alan Michael Huberman, but it was difficult to
> follow since it did not give examples that I can relate to. I would
> really appreciate it if someone could point me to information that
> will help me learn more about this topic.
>
> Thanks,
> Oliver
> ________________________________________________________________
> *Come to IxDA Interaction08 | Savannah*
> February 8-10, 2008 in Savannah, GA, USA
> Register today: http://interaction08.ixda.org/
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help
>
At 9:38 AM +0530 11/15/07, Kunal Kapoor wrote:
>What data do you want to analyze (Qualitative).
>
>There are two schools of thought, one the purists, who do not like doing
>things like t tests with Likert scale ratings, and another school of
>thought, where people do such tests.
>
>What exactly is the data you have and what is it that you want to analyze?
>
>- Kunal.
Just as a point of information, I'll go those purists one better and
point out that technically there is no such thing as "qualitative
data". Data refers to the measurable and qualitative information is
by definition not quantifiable.
This actually may be an important point for you in understanding how
to go about discussing and assessing qualitative information. The
minute you start talking about analyzing qualitative data you've
moved from the qualitative information itself into the realm of
quantitative elements which have been observed to be associated with
the qualities themselves.
--
----------------
Katie Albers
katie at firstthought.com
Hi Oliver,
one well-known technique used for analysing qualitative data is called
"Grounded Theory". You can find a good summary of the method here:
http://www.scu.edu.au/schools/gcm/ar/arp/grounded.html
(it may also be covered in your book).
Don't be scared by all the terminology that goes with this process (it
comes from a Social Research background). Try to understand the spirit
of the technique and apply it in whatever time you have. I'm no expert
either, but I used it quite succesfully for a research project to
analyse user's attitues on digital photography tools & equipment. If
you want to take a look at the report you can download it at:
http://futureshape.net/cms/uploads/documents/TheJourneyOfADigitalPhoto.pdf
(8MB PDF, sorry, you can skip to the end of the 3rd chapter)
Let me know how it goes & if you need any more ideas!
Good luck,
Alex
On Wed, 14 Nov 2007 21:20:42, Oliver Green <oliverhci at gmail.com> wrote:
> Good catch. I am analyzing notebook diaries recording thoughts and
> observations during the usage of the system and audio interviews
> conducted at the end of the session.
>
> Thanks,
> Oliver
>
>
> . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Posted from the new ixda.org
> http://www.ixda.org/discuss?post=22609
>
>
>
> ________________________________________________________________
> *Come to IxDA Interaction08 | Savannah*
> February 8-10, 2008 in Savannah, GA, USA
> Register today: http://interaction08.ixda.org/
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help
>
So,
you can 'cluster' feedback under broad categories (for a single strategy -
then, analysis can then be done using Multidimensional scaling (MDS) and
Linear regression, to identify the most important categories on which
immediate attention is needed, if resource allocation is a constraint .. I
would assume that if you follow UCD, you would already establish marginal
and ideal values for specific metrics associated with the product, something
of that nature can also be incorporated for usability issues .. cause MDS is
'hectic').
if you are collecting preference ratings for 2 or more different strategies,
you can do a t-test (this would differ on what kind of test you incorporate,
depending on within or between subjects design) to see which is the more
preferred strategy (indicated through statistical significance).
again in your response you do mention WHAT you are doing, and I am trying to
tell you HOW you might do things,
I am not sure WHAT you are MEASURING, to be able to provide more input on
the ANALYSIS side.
hope this helps.
sincerely
kunal
On Wed, 14 Nov 2007 21:20:42, Oliver Green <oliverhci at gmail.com> wrote:
>
> Good catch. I am analyzing notebook diaries recording thoughts and
> observations during the usage of the system and audio interviews
> conducted at the end of the session.
>
> Thanks,
> Oliver
>
>
> . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Posted from the new ixda.org
> http://www.ixda.org/discuss?post=22609
>
>
> ________________________________________________________________
> *Come to IxDA Interaction08 | Savannah*
> February 8-10, 2008 in Savannah, GA, USA
> Register today: http://interaction08.ixda.org/
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help
>
In your email you mention -- "usage of the system" ....
do you have preference ratings for a previous version of the product, and
the current version, or is this the ONLY version you are trying to test ?
if you do, then you can use a t - test
if not,
forming clusters for the most important issues would be one way to do
some meaningful analysis
- kunal
On Wed, 14 Nov 2007 21:20:42, Oliver Green <oliverhci at gmail.com> wrote:
>
> Good catch. I am analyzing notebook diaries recording thoughts and
> observations during the usage of the system and audio interviews
> conducted at the end of the session.
>
> Thanks,
> Oliver
>
>
> . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Posted from the new ixda.org
> http://www.ixda.org/discuss?post=22609
>
>
> ________________________________________________________________
> *Come to IxDA Interaction08 | Savannah*
> February 8-10, 2008 in Savannah, GA, USA
> Register today: http://interaction08.ixda.org/
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help
>
I've used affinity diagram for something like this. The Wikipedia
entry has a good primer if you are not familar with it. You can also
buy the book on contextual Design for the background.
If you can do this in the workshop setting, it is a great technique
to get the entire team involved with the analysis of the data.
Delivering a final report, to me, is of limited value. Getting folks
to spend a day emurst in the user data is powerful.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=22609
Kunal asked "do you have preference ratings for a previous version of
the product, and the current version, or is this the ONLY version you
are trying to test ?"
This is the only version that we are trying to test.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://gamma.ixda.org/discuss?post=22609
One process our team uses to distill large volumes of interview
feedback into useful recommendations for our UI is a modified
keyword/cluster/textual analysis. Our technique is a modification of
a couple methods of rhetorical analysis so, unfortunately, I don't
have a good source to point you to.
To set up we use a basic spread sheet with rows of users and columns
of keywords and functions pertinent to our UI. For instance if we
were reviewing feedback from a music player application our keywords
would be things like Song, Playlist, etc. We document the users
phrase or thought associated with each keyword. Things like, User 1
commented: The playlist is hard to use, User 2 commented: I expected
to be able to sort the playlist by clicking on the heading, etc.
After everything is sorted into more manageable chunks we take all of
the text about the playlist and see how many people said it was hard
to use, versus how many said it was an improvement, etc. Patterns
and themes will emerge that can help you see the trends in how a
group is thinking or feeling about aspects of an application. This
is not really an analysis technique but a process to get from point A
(huge amount of material to go through) to point B (understanding of
what users think).
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=22609