Eye-Tracker software/hardware recommendations

14 Aug 2009 - 4:31am
6 years ago
72 replies
3129 reads
William Hudson
2009

Kristen -

I attended a pro vs cons debate on eye-tracking a few months ago in
London. On the 'cons' side was Kara Pernice who is the MD at NNG. Her
main thrust was that eye-tracking was largely irrelevant for most
usability work, particularly given the expense.

I have since had a couple of conversations on this with heavy
eye-tracking users. Both confessed that it really was overkill for most
usability studies and that you did have to be experienced in order to
interpret the results correctly (so it wasn't something you would use
casually). On the plus side, it does some to be a good way to engage
technology-oriented clients!

Regards,

William Hudson
Syntagm Ltd
Design for Usability
UK 01235-522859
World +44-1235-522859
US Toll Free 1-866-SYNTAGM
mailto:william.hudson at syntagm.co.uk
http://www.syntagm.co.uk
skype:williamhudsonskype

Syntagm is a limited company registered in England and Wales (1985).
Registered number: 1895345. Registered office: 10 Oxford Road, Abingdon
OX14 2DS.

Confused about dates in interaction design? See our new study (free):
http://www.syntagm.co.uk/design/datesstudy.htm

12 UK mobile phone e-commerce sites compared! Buy the report:
http://www.syntagm.co.uk/design/uxbench.shtml

Courses in card sorting and Ajax interaction design. London, Las Vegas
and Berlin:
http://www.syntagm.co.uk/design/csadvances.shtml
http://www.syntagm.co.uk/design/ajaxdesign.shtml

> -----Original Message-----
> From: new-bounces at ixda.org [mailto:new-bounces at ixda.org] On Behalf Of
> Kristen
> Sent: 13 August 2009 11:23 AM
> To: discuss at ixda.org
> Subject: [IxDA Discuss] Eye-Tracker software/hardware recommendations
...

Comments

22 Aug 2009 - 4:28am
Guy Redwood
2009

I need to brush up on my 'dealing with phobics' skills. A lot of the
positions against eye tracking are unreasonable. There is a clear
correlation between what people look at and what they comprehend. Why
would anybody deny it? I've never been very good at reading things
I've not looked at!

At the last eye tracking conference in Frankfurt we discussed the
issues with what poor research was doing to the reputation of the eye
tracking industry. I didn't realise that Mr Spool had played a part
in that over the last 15 years.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from ixda.org (via iPhone)
http://www.ixda.org/discuss?post=44684

22 Aug 2009 - 8:51am
Christian Crumlish
2006

On Fri, Aug 21, 2009 at 7:28 PM, <guy at simpleusability.com> wrote:

> I need to brush up on my 'dealing with phobics' skills. A lot of the
> positions against eye tracking are unreasonable. There is a clear
> correlation between what people look at and what they comprehend.

A 1:1 correlation? What sort of correlation? I'm not sure I've heard anyone
on this thread deny that comprehension is related to seeing which is related
to gazing, but the devil is in the details, no?

> Why
> would anybody deny it? I've never been very good at reading things
> I've not looked at!
>

This sounds like a straw-man argument to me.

At the last eye tracking conference in Frankfurt we discussed the
> issues with what poor research was doing to the reputation of the eye
> tracking industry. I didn't realise that Mr Spool had played a part
> in that over the last 15 years.
>
>
I'm sort of amazed there are eyetracking conferences and an eyetracking
industry. Is there a cardsorting industry?
-x-

--
Christian Crumlish
I'm writing a book so please forgive any lag
http://designingsocialinterfaces.com

22 Aug 2009 - 11:31am
Jared M. Spool
2003

On Aug 22, 2009, at 2:28 AM, guy at simpleusability.com wrote:

> There is a clear
> correlation between what people look at and what they comprehend.

No. No there isn't. If there's a correlation, it's definitive unclear.

Please, clear it up for all of us.

> At the last eye tracking conference in Frankfurt we discussed the
> issues with what poor research was doing to the reputation of the eye
> tracking industry.

Produce the "good research" Guy. I'd love to see it.

Jared

22 Aug 2009 - 5:12pm
Todd Warfel
2003

On Aug 21, 2009, at 6:22 PM, Nick Gould wrote:
> Seems that, given your professional impermeability relating to this
> issue, you could just leave well enough alone; give your opinion
> when asked but otherwise respect the right of others to run their
> businesses as they see fit. Anyway...

Nick, I don't have a problem with someone running their business as
they see fit as long as it doesn't impact the field I work in. And
herein lies the crux of the problem with your statement.

You see, as a designer and UX professional, I'm part consultant and
part educator to my clients and this field. As a consultant, my role
is to provide services to my client that have a measurable impact on
their business. As an educator, my duty is to educate them ethically
about what our field provides.

Why is honesty, integrity, and ethics so hard to come by? Perhaps the
shiny color of that gold coin is more inviting that the value of doing
real and meaningful work.

I take pride in my field, my work, the service this field can provide
to the world, what we can contribute, and the legacy we can leave
behind. This is why I personally take issue with things like this.
Eyetracking doesn't really provide any value other than to show some
fancy visualization heat maps on screen. That's all it does.

Yeah, it's impressive to see those heat maps. I love looking at them.
But that's the only true value—visual aesthetics. It doesn't really
tell you anything about why anyone does anything. Making that
inference is a HUGE unsubstantiated leap. The claims I typically see
made through ET in my view are unethical and unsupportable.

Instead of trying to find a solution that ET solves, which to date and
in 15 years in this field, I've not seen one, we should be focusing
our efforts on existing research methods, or developing new ones, that
actually do provide value, provide quality data, and from which we can
make reliable inferences with integrity.

When I've pressed ET advocates on the reliability of the data they
produce and the reliability of the inferences they're making based
solely on ET data, they buckle like a house of cards. Call a spade a
spade. It's about as scientifically valid as snake oil.

Cheers!

Todd Zaki Warfel
Principal Design Researcher
Messagefirst | Designing Information. Beautifully.
----------------------------------
Contact Info
Voice: (215) 825-7423
Email: todd at messagefirst.com
AIM: twarfel at mac.com
Blog: http://toddwarfel.com
Twitter: zakiwarfel
----------------------------------
In theory, theory and practice are the same.
In practice, they are not.

22 Aug 2009 - 5:15pm
Todd Warfel
2003

On Aug 22, 2009, at 9:51 AM, Guy wrote:

> At the last eye tracking conference in Frankfurt we discussed the
> issues with what poor research was doing to the reputation of the
> eye tracking industry.

Perhaps you should've been discussing the harm that eye tracking does
to real research.

Cheers!

Todd Zaki Warfel
Principal Design Researcher
Messagefirst | Designing Information. Beautifully.
----------------------------------
Contact Info
Voice: (215) 825-7423
Email: todd at messagefirst.com
AIM: twarfel at mac.com
Blog: http://toddwarfel.com
Twitter: zakiwarfel
----------------------------------
In theory, theory and practice are the same.
In practice, they are not.

23 Aug 2009 - 9:04am
Caroline Jarrett
2007

On another list recently, Carolyn Snyder pointed out that there are two
purposes to user research:

1. Finding out about the product
2. Changing the product

(I paraphrase).

Jared seems to be focusing strongly on point 1. I somewhat sympathise with
his point of view, in that I've not found that the eye-tracking stuff adds
greatly to what I can find out from an ordinary observational test without
eye-tracking.

I'd prefer to focus somewhat on point 2, which to me is about influencing
stakeholders to make them want to make changes based on what I've found. I
have found that eye-tracking stuff can be rather helpful, in some
circumstances, in helping stakeholders to understand what we've found and
persuade them to act on it.

If your work consists solely of the finding out aspect, then probably I'd
agree that you don't need eye-tracking.

If you are stakeholder/decision-maker, then I'd also probably agree that you
don't need eye-tracking.

For the rest of us, who want to make changes but need to influence other
people to do so: it can be helpful. It's another tool in the toolbox, and I
don't see why we shouldn't use it just because some people don't feel the
need to use it.

And I declare an interest: I have used data from eye-tracking in my talks on
forms. I find that the illustrations help attendees to see what I've seen.

Best
Caroline Jarrett
www.formsthatwork.com
"Forms that work: Designing web forms for usability".

24 Aug 2009 - 9:00pm
Will Hacker
2009

"I'm starting too see that both camps are not open to persuasion"

@ritchielee Have to disagree. This thread has changed the way I think
about eye tracking. I'm not saying I'm now for or against it with
any fervor, but am thinking about it more critically. And isn't that
the point of these discussions.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=44684

25 Aug 2009 - 7:44am
Guy Redwood
2009

Todd said:
"Yeah, it's impressive to see those heat maps. I love looking at
them. But that's the only true value%u2014visual aesthetics. It
doesn't really tell you anything about why anyone does anything."

Indeed.

That's why the in-depth understanding comes from using the
eyetracking data in a retrospective review session with the users.
When you play back eye tracking to the user, they tell you why they
did things.

We don't sit there looking at heatmaps, making things up.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=44684

25 Aug 2009 - 8:57am
Jared M. Spool
2003

On Aug 25, 2009, at 5:44 AM, Guy Redwood wrote:

> That's why the in-depth understanding comes from using the
> eyetracking data in a retrospective review session with the users.
> When you play back eye tracking to the user, they tell you why they
> did things.
>
> We don't sit there looking at heatmaps, making things up.

No. You make things up first, then badger the user into admitting
that's what they were thinking by showing them irrelevant eye tracking
footage.

That's my opinion of retrospectives with eye tracking. (PEEP method
and others. PEEP claims its innovative, but we were doing
retrospectives with eye tracking in 1993.)

Don't get me wrong. Retrospectives are a fabulous tool and I love
them. It's the introduction of the eye tracking data into the
retrospective process that distorts it past meaning.

Jared

25 Aug 2009 - 9:21am
Guy Redwood
2009

Jared, we don't badger and we don't make things up.

Users are very relaxed and chat freely in our retrospective sessions.
That's the point.

Are you assuming that we need to badger users because that's what
you do in your think aloud sessions? Is that because the user is
trying to do something, then trying to think about why they are doing
something, then trying to verbalise what they are thinking - oh and
also trying to remember what they had set out to do in the first
place.

Here's a video of some retail testing.
http://www.youtube.com/watch?v=38w95lKFIWc
What do you find irrelevant about the eye tracking?

I've said it before but I'll repeat it here - You have an open
invitation from us to come and see how we do things and hopefully
take a fresh look at eye tracking. You've clearly had some bad
experiences.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=44684

25 Aug 2009 - 2:11pm
Nickgould
2009

Wow, just as I was thinking that this thread was getting a little
tiresome and unproductive, I discovered that it actually started in
2005!

http://www.ixda.org/search.php?tag=eyetracking

Jared - it's really not fair as you have had way more practice
attacking eyetracking than I have had defending it. I need to go
train some more...

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=44684

25 Aug 2009 - 5:25pm
Jared M. Spool
2003

On Aug 25, 2009, at 7:21 AM, Guy Redwood wrote:

> Here's a video of some retail testing.
> http://www.youtube.com/watch?v=38w95lKFIWc
> What do you find irrelevant about the eye tracking?

I didn't see anything in the video that surprised me or informed me
about users interacting with the Amazon site.

When I read the "full commentary" on http://is.gd/2ytGQ, I didn't see
anything that was shown by the eye tracking that wouldn't have been
shown through a traditional test. (I know this because I've watched
about 400 shoppers on Amazon over the last 13 years, so I'm well aware
of what we can and can't learn from traditional tests. Oh, by the way,
I've done eye tracking studies on Amazon a few years back. Nothing you
show here is any different that what we saw then.)

Where was the retrospective interview? I'd like to see (and hear) that.

You're going to have to do better than this to convince me that eye
tracking adds any value.

Jared

25 Aug 2009 - 5:48pm
Jared M. Spool
2003

On Aug 23, 2009, at 9:04 AM, Caroline Jarrett wrote:

> On another list recently, Carolyn Snyder pointed out that there are
> two
> purposes to user research:
>
> 1. Finding out about the product
> 2. Changing the product
>
> (I paraphrase).
>
> Jared seems to be focusing strongly on point 1. I somewhat
> sympathise with
> his point of view, in that I've not found that the eye-tracking
> stuff adds
> greatly to what I can find out from an ordinary observational test
> without
> eye-tracking.

Yes. That's where I've been focusing because that's where I think the
egregious practices are in place.

>
> I'd prefer to focus somewhat on point 2, which to me is about
> influencing
> stakeholders to make them want to make changes based on what I've
> found. I
> have found that eye-tracking stuff can be rather helpful, in some
> circumstances, in helping stakeholders to understand what we've
> found and
> persuade them to act on it.

I haven't focused here because I agree: it's a great piece of
theatrical stagecraft to demonstrate where the users' eyes bounces
around to as you add a commentary that says what the problems with the
design are. Add a little pyrotechnics and a belly dancer, you'd have a
complete show ready to compete with Cirque de Soleil.

>
> If your work consists solely of the finding out aspect, then
> probably I'd
> agree that you don't need eye-tracking.
>
> If you are stakeholder/decision-maker, then I'd also probably agree
> that you
> don't need eye-tracking.
>
> For the rest of us, who want to make changes but need to influence
> other
> people to do so: it can be helpful. It's another tool in the
> toolbox, and I
> don't see why we shouldn't use it just because some people don't
> feel the
> need to use it.

I agree.

There is a problem that we haven't addressed though:

What happens when they notice it's fake?

For example, in presentation-to-influencers #1, we say, "As you can
see, the user fixated on the content, which means that they were
confused. Therefore, we need to fix them content presentation."

Yet, months later in presentation-to-influencers #2, we say, "As you
can see, the user fixated on the content, which means they were
completely satisfied and absorbed by it. Therefore, we think the
content presentation works perfectly."

What do we do when they say, "But before, you said fixation was bad.
Now you say fixation is good. Which is it?"

Do we say, "Well, actually, we don't know what fixation means, so we
just tell you it means whatever serves our purposes at the time"? Is
that really the right way to treat our clients?

While I like the razzle dazzle of the eye tracker, I tend to go with
the more honest and straightforward approach of involving them in the
study and letting them come to their own conclusions. That's my
approach and I know it's not generally accepted as a best practice.

Jared

25 Aug 2009 - 8:18pm
Nickgould
2009

Jared, you are selling Caroline's point short... how about this
scenario:

We report that test participants asked to locate the search box
looked in the upper right corner for it. They told us that this is
where they expected it to be and the eyetracking confirms that this
is where they looked for it.

So, yes, the ET lent further support to a talk aloud finding. For
some clients - rightly or wrongly - this strengthens their confidence
in the results. That's not razzle dazzle, it's just additional,
consistent feedback. They "said" this and they "did" this. Why is
it any different from reporting where they clicked?*

Sure, it's optional - I don't think anyone claims ET replaces talk
aloud or that it's even necessary for a good study. But it can be a
valid, additional tool (in the right hands) for helping clients to
feel comfortable about the research results.

NG

*As an aside, I think it's interesting that many of your arguments
against eyetracking could also be leveled against clickstream
analysis / clickmaps, etc... I am amazed at how willing clients are
to believe that this data is meaningful on its own.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=44684

26 Aug 2009 - 3:33am
Guy Redwood
2009

Jared - After reviewing the video, I was hoping you would expand your
thoughts on the validity of eye tracking data and ouija boards and
correlation to thought processes.

You've suggested that fixations don't offer any
evidence/insight/it's false etc.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=44684

26 Aug 2009 - 10:34am
Jared M. Spool
2003

On Aug 26, 2009, at 1:33 AM, Guy Redwood wrote:

> Jared - After reviewing the video, I was hoping you would expand your
> thoughts on the validity of eye tracking data and ouija boards and
> correlation to thought processes.
>
> You've suggested that fixations don't offer any
> evidence/insight/it's false etc.
>

And what in the video was supposed to show me that it offered any
insights? I missed them. I just saw a repeat of findings that I knew
without having to watch the eye tracking display, but just working
with users in a traditional usability test. Where was the thing I'd
only discover watching eye movements?

Maybe I'm just too dense to understand this eye tracking stuff,
despite the hundreds of hours I've sat in studies watching people
interact with them?

It's clear that I'm not smart enough for this new technology. I think
that's the message we can take from this seemingly-endless thread.

Jared

Jared M. Spool
User Interface Engineering
510 Turnpike St., Suite 102, North Andover, MA 01845
e: jspool at uie.com p: +1 978 327 5561
http://uie.com Blog: http://uie.com/brainsparks Twitter: @jmspool

26 Aug 2009 - 10:49am
Jared M. Spool
2003

On Aug 25, 2009, at 6:18 PM, Nick Gould wrote:

> Jared, you are selling Caroline's point short... how about this
> scenario:
>
> We report that test participants asked to locate the search box
> looked in the upper right corner for it. They told us that this is
> where they expected it to be and the eyetracking confirms that this
> is where they looked for it.

But that's my point! Who cares if they looked in the upper right for
the search box? The real question that will make a difference in the
performance is, "Did the user find the search box?" Our studies show
that, while users expect the search to be in the upper right, they are
not hampered in any meaningful way if it's anywhere else prominent on
the page. (See http://is.gd/2zZM1.)

Now, maybe you reached for the wrong example here. If you want to try
again, I'm all for it.

But there is something we can learn from this example. Because you
were focusing on the granularity of "where does the user look" and not
"what does the user need", you were focusing on the wrong problem to
solve. Moving the search box from whereever it is to the upper right
would not solve any problem. The data you collected forced you to
focus on the wrong problem, which would commandeer resources to
generate a result that probably won't make the design any better.

I'm not sure which point of Caroline's you think I'm selling short.
Was it this one?

> Jared seems to be focusing strongly on point 1. I somewhat
> sympathise with
> his point of view, in that I've not found that the eye-tracking
> stuff adds
> greatly to what I can find out from an ordinary observational test
> without
> eye-tracking.

Because I don't think I sold that point short at all!

> So, yes, the ET lent further support to a talk aloud finding. For
> some clients - rightly or wrongly - this strengthens their confidence
> in the results. That's not razzle dazzle, it's just additional,
> consistent feedback. They "said" this and they "did" this. Why is
> it any different from reporting where they clicked?*

It's only consistent feedback because you decided to make it
consistent. You could just as easily say that those eye movements are
involuntary and not really indicative of how someone succeeds or fails
with the design.

As you (or someone in this thread) stated, eye tracking records
unconscious behaviors. Applying meaning to unconscious behavior is a
difficult road to go down, because the odds of applying the wrong
meanings are very high.

> Sure, it's optional - I don't think anyone claims ET replaces talk
> aloud or that it's even necessary for a good study. But it can be a
> valid, additional tool (in the right hands) for helping clients to
> feel comfortable about the research results.

Yah, you keep repeating this. Still waiting to see what that
additional value is.

If ET doesn't replace traditional stuff, why bother with it? The
traditional stuff is easier, cheaper, better understood, proven to
work, and more reliable.

Jared "The Proof is in The Pudding" Spool

27 Aug 2009 - 9:06am
Elizabeth Buie
2004

Here's something that's guaranteed to make Jared crazy:
"10 Web Form Design Guidelines Based on Eyetracking"

http://www.smileycat.com/miaow/archives/001750.php

:-)

Not only were these guidelines around before eye tracking, one of them is even
wrong. (I am speaking of the one about left-aligning labels; and the reason
is that it depends on the purpose of the form and the lengths of the labels.)

Elizabeth
--
Elizabeth Buie
Luminanze Consulting, LLC
tel: +1.301.943.4168
www.luminanze.com
@ebuie

27 Aug 2009 - 11:43am
ritchielee
2009

"eye tracking provides much needed razzmatazz to impress clueless
people.. who don%u2019t understand usability." - Jakob Neilsen

@will. That's cool. I think Jakob nails it here: it can be used as a
visual to all the good stuff you already eek out with proven usability
methods.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=44684

27 Aug 2009 - 1:21pm
Nickgould
2009

Those following this thread might be interested in a study just
published in the JUS that used eyetracking to compare newspaper
websites and tv news websites.

http://www.upassoc.org/upa_publications/jus/2009august/gibbs1.html

To me, this is interesting not so much for its conclusions (which,
frankly, amount to very little) as for the myriad citations to
scholarly research going back many years regarding various issues
touched on in this discussion. Anyone with the time or inclination
might like to peruse some of this previous research.

At the same, the authors of this study concede that "there has been
insufficient work that links usability with eyetracking results."

As previously, the situation is as clear as mud.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=44684

3 Sep 2009 - 5:09pm
Ali Naqvi
2008

According to the research below, eye tracking with retrospective reporting "can be more insightful and beneficial to the usability tester than a conventional think-aloud protocol produced concurrent to primary task processing."

http://www.bcs.org/upload/pdf/ewic_hc07_lppaper13.pdf

Syndicate content Get the feed