What is the best way to merge qualitative and quantitative data?

1 Mar 2010 - 3:42pm
4 years ago
9 replies
3415 reads
iheartusers
2010

Good Afternoon IXDA…


I could really use your expertise to help me answer this question…


I am working with an ecommerce client on ways for them to integrate qualitative data (surveys, usability testing, interviews, etc.) with quantitative data (google analytics, coremetrics, keynote performance data and even SEO data). We are really looking for the interplay of these two data types.

We need to integrate the reporting and come up with KPIs that help management see and monitor the big picture. The idea isn’t to report on Customer Satisfaction separate from Conversion, but to rather bring these types of KPIs together into a management dashboard.

The most import thing is that we are able to derive actionable insights from our work.

So what is the best way to merge qualitative and quantitative data?
Any suggestions, examples, best practice references would be OHH so appreciated.

 

Best Regards,

Carrie Sitz Trieglaff

carrie@iheartusers.com

The Happy User Researcher

Twitter: iheartusers

Comments

1 Mar 2010 - 3:46pm
iheartusers
2010

Thanks for your comments.  If you have any further comments or want to contact me directly.  I can be reached at carrie@iheartusers.com

Best Regards,

Carrie

1 Mar 2010 - 4:52pm
Loren Baxter
2007

Hi Carrie,

Livia Labate gave a pretty good talk on this at IxD10. http://www.ixda.org/resources/livia-labate-ceci-n-est-pas-une-kpi

1 Mar 2010 - 4:59pm
iheartusers
2010

Thanks Loren,

Do you know if my question was also posted to the list or if there is an additional step to post it to the email list??

(I am pretty new to IXDA)

Thanks,
Carrie

1 Mar 2010 - 5:22pm
Loren Baxter
2007

The email list isn't working at the moment - they've relaunched the whole site over the weekend, so it's still in the process. It will probably be working again in a day or two.

1 Mar 2010 - 6:24pm
iheartusers
2010

thanks for taking the time to share that with me. 

btw - the site looks nice.

 

Carrie

2 Mar 2010 - 12:05pm
timsheiner@gmail.com
2007

Carrie;

'Best' is a loaded term but I think you really only have three choices. 

  1. You can display the subjective data as some kind of editorial summary.
  2. You can report subjective data as direct quotes or observations
  3. You can transform subjective data into objective data. 

 

By transform I mean create some way to score the subjective data, and report those scores.  In surveys you can ask users to create the score themselves (e.g. 'rank your experience on a score of 1-5').  For interviews, you can ask the interviewer to create scores (e.g. 'how satisfied was this customer on a scale of 1-5?'). You can also combine several scores arithmetically into a KPI.  You can also create a KPI by combining scores with hard data (e.g. Actual Satifaction = reported satisfaction * actual renewal rate).

Personally, I have had success with a combination technique.  I report subjective data as a written, editorial summary of findings, in which I also include direct quotes and report scores that I've created.  This is, unfortunately, a lot of work to do, and I have never tried to deliver the result in an automated way to a dashboard.

Tim

2 Mar 2010 - 4:28pm
Edo A. Elan
2004

I try to redefine the issue - from "quantitative vs. qualitative", to "web vs. business measurement".

Suppose a website gathers leads on the web; these leads may mature into transactions. Such web-related ("quantitative") analysis may show some UI enhancemnents to be neutral (waste of time) or even negative (less clicks and less conversions).

But what if some of the converted leads mature into customers, and you measure lifecycle length, transaction amount, retention? what if your conversions stay longer, or are easier to process? These are all quantitative too - once you continue the tracking from the web into the business cycle.

This isn't just an off-the-top-my-head notion, but a way to show managers that interaction isn't just measured in number of clicks, but it has the power to improve the quality of the conversions. 

The reason why web stats are dangerous is that, basically, they stop at the middle of the business process. If all you look for are web stats then you might as well set up a big red button saying "You have been chosen as our 1 millionth customer, click here to receive a free iPod". You'll get many clicks, but little business - and this is a quantitative measure.

I found that sales and customer service people are my allies when such questions are discussed. Business intelligence systems can be just as challenging to install as web measurements systems, especially if the organization isn't already interested in those concepts - but all you need is one friendly contact who has access to long term data. 

Put in a different way - as an alternative to web tracking, you may want to analyse the characteristics and preferences of your best customers and show what they're influenced by.

5 Mar 2010 - 7:09pm
iheartusers
2010

Thanks for all of your responses.  If anyone is interested in what we ultimately recommend and the results, drop me a note and I will be sure to share with you a shell and case study.  If anyone has any additional suggestions on how to merge qualitative and quantitative data, I would love to hear them.  Thanks again IXDA!

Best Regards,
Carrie Sitz Trieglaff   |  
carrie@iheartusers.com   |  
The Happy User Researcher   |  
Twitter: iheartusers   |

26 Mar 2010 - 3:01pm
Joshua Gross
2006

I don't have interaction guidelines, but I wrote this review with a
colleague about an excellent overview on mixed methods in research: http://bulletin.sigchi.org/2005/may/review-of-mixing-methods-in-psychology/

vr Joshua

On Mar 2, 2010, at 12:26 AM, iheartusers wrote:

> Good Afternoon IXDA… > > I could really use your expertise to help me answer this question… > > I am working with an ecommerce client on ways for them to integrate
> qualitative data (surveys, usability testing, interviews, etc.) with
> quantitative data (google analytics, coremetrics, keynote
> performance data and even SEO data). We are really looking for the
> interplay of these two data types.We need to integrate the reporting
> and come up with KPIs that help management see and monitor the big
> picture. The idea isn’t to report on Customer Satisfaction separate
> from Conversion, but to rather bring these types of KPIs together
> into a management dashboard. The most import thing is that we are
> able to derive actionable insights from our work. > > So what is the best way to merge qualitative and quantitative data? > Any suggestions, examples, best practice references would be OHH so
> appreciated. > > > > Best Regards, > > Carrie Sitz Trieglaff\ > > carrie@ihearusers.com [1] > > The Happy User Researcher > > Twitter: iheartusers > > (((Ple

Syndicate content Get the feed