Software for Multivariate Usability Testing

14 Oct 2011 - 11:30am
3 years ago
8 replies
8743 reads
Powers
2008

Does anyone have any suggestions for software that allows for multivariate testing? I've got a project that have numerous paths a user can take and standard click throughs wont cut it. Based on user selections from initial steps the later steps may change and the software would need to allow for this. 

context: http://en.wikipedia.org/wiki/Multivariate_testing

Comments

14 Oct 2011 - 1:02pm
aschechterman
2004

JP, Would you need something as powerful as SPSS or SAS? They offer basic packages but I've found they can be too much of a good thing for a lot of the UX qual + quant challenges we address. When I was in healthcare-medicine I got my own copy of SPSS and now use perhaps 2% of it, a few times a year. The R-Project software is free, works well . . . http://www.r-project.org/. Would PC Calc be too lightweight? http://www.pcalc.com/english/about.html - Andrew

14 Oct 2011 - 1:05pm
okosoft
2010

Hi! What is multivariate testing?

I do eye tracking test. I have mirametrix eye tracker with my own software and tobii eye tracker. But I don't understand what is multivariate....

16 Oct 2011 - 2:06am
Yohan Creemers
2008

Andrei,

Multivariate testing is an effectiveness test of more than one design variable, often tested in a live environment. In the case of websites 50% of the visitors will see one version of the page design, the other half will see another version of the same page. By analyzing the visitor clicking or navigation behavior the effectiveness of the design alternatives is determined.

- Yohan

17 Oct 2011 - 6:18am
Rich Gunther
2010

The software developed by my company, Ovo Studios, can probably support this.  We allow you to configure any number of experimental treatments and assign a different to different users.  You could vary task order, which tasks/scenarios you want the user to work with, which URL you want to present to the user as a stimulus, etc. 

Outside of this scripted approach, you can manually run users through tasks in any order you wish, which should support your "user selections from earlier steps influence later steps" use case.

Check out either Ovo Solo or Ovo Logger at www.ovostudios.com.

19 Oct 2011 - 4:53am
Evan Green
2011
It appears that you have gotten some great answers here. I hope that your project turns out better than you expected and that you might have the time to share with us the details after it is all over.

 

wager slowly

Five Card Stud at this online casino is just one of the poker games that are here for your gambling and entertainment pleasure but you have to wager slowly.

7 Dec 2011 - 3:27pm
wusatiuk
2011

Depending on the budget, but generally you can do this with some tools like Google Analytics + Google Website Optimizer as well.

9 Dec 2011 - 1:09am
Alfonso de la Nuez
2009

As usual, it all depends on your research goals. While pretty powerful, multivariate testing does not typically offer much usability data. That's my personal experience. It tells you which design variable performed best (i.e. sold more, got more registrants, etc.), but it does not tell you the fundamental usability/UX question: WHY. That's why many researchers combine multivariate with usability testing (and web analytics, surveys, ethnographic studies, etc.). 

I'd offer you our unmoderated remote user testing methodology and products at UserZoom. You can certainly test 2 design versions over hundreds of users and collect usability metrics, success ratios, click heatmaps, clickstreams, and responses to a follow up (post task) questionnaire. More info at www.userzoom.com

Good luck!

9 Dec 2011 - 10:05am
GeoffWill
2010

Multi-variate analysis is a multiple ANOVA. Most usability studies do not use sufficient participants and groups to be suitable for ANOVA. In most cases researchers better spend their usability dollars on several studies rather than on studies with 15+ participants per group. One might ask what is the nature of your study that it requires multi-variate analysis? Is this really a relevant technique for the variables of interest?

Geoff Willcher

-----Original Message----- From: Alfonso de la Nuez Sent: Friday, December 09, 2011 2:56 AM To: gwbando@msn.com Subject: Re: [IxDA] Software for Multivariate Usability Testing

As usual, it all depends on your research goals. While pretty powerful, multivariate testing does not typically offer much usability data. That's my personal experience. It tells you which design variable performed best (i.e. sold more, got more registrants, etc.), but it does not tell you the fundamental usability/UX question: WHY. That's why many researchers combine multivariate with usability testing (and web analytics, surveys, ethnographic studies, etc.).

I'd offer you our unmoderated remote user testing methodology and products at UserZoom. You can certainly test 2 design versions over hundreds of users and collect usability metrics, success ratios, click heatmaps, clickstreams, and responses to a follow up (post task) questionnaire. More info at www.userzoom.com

Good luck!

Syndicate content Get the feed