How to hire a good IxD (RE: Eye tracking, how valuable is it?)

7 Jul 2006 - 10:59am
8 years ago
6 replies
559 reads
Dave Malouf
2005

> Care to outline exactly how one goes about hiring a *good* UI
> designer? ;-)

First some thoughts on eye-tracking of my own.
I have found the case studies I have read on eye-tracking to be quite
outstanding. What is gleaned from them in my mind is applicable across many
a domain. Understanding what part of layouts and text people are ignoring
tells a designer a lot about their design and I can see how this information
CAN be useful.

<caveats>
1. you have the time and budget to do the analysis of the results. I know
for myself I don't even bother with video analysis b/c I don't even have
time for that. My usability tests are very qualitative and anything more
statistical is out of my reach. That isn't to say that statistical
quantitative analysis isn't appropriate when the resources are available,
but resourcing is important.

2. eye-tracking seems to be most useful when the pool is deep. You need to
have more users to get to the greatest level of fine tuning to derive
conclusions.

3. conclusions seem to only come at the extremes when I read eye-tracking
case studies. It always seems to be a little "too perfect".
</caveats>

My conclusion though is that for most people with limited resources and
budgets is that a good seasoned designer with qualitative skills goes
further than adding in eye-tracking and doing proper analysis on it.

Now, that leads to Jared's question ... How do you hire the right designers?

I have had VERY mixed results in my hiring over the last few years. There is
so much to hiring a good designer and often HR departments are of very
little help because they have little to no experience working with designers
and thus don't understand that they are not like the rest of the
development/production team. (I'm an insider, but have experience hiring as
a consultant as well.) While on the consulting/agency side of things, I
found that internal recruiters are a lot more helpful.

What does it mean for a recruiter in your HR dept to be helpful:
1. They understand that design is not making pretty things
2. They understand the cultural environment for designers in the
organization and the repercussions that has on hiring designers as a
different cultural career path than developers and other business
professionals
3. They create pay-scale and bonus plans that speak to the creative class
4. They understand the career paths for designers are very different than
for other professionals. I.e. very few organizations have non-management
career paths in their organizations. Yes I know there are some that do, but
they are the exception that get it. What is funny is often there have this
career path in engineering--i.e. the principal engineer--but don't know how
to engage that path within the design organization. Even agencies have this
problem.

After you deal w/ the HR issues, finding a GOOD designer is probably one of
the hardest things to do. How do you judge that? How many of the 100's of
resumes you get are just a waste of time?

So without getting into the whole, what should a portfolio look like debate,
I want to just concentrate on something that seems to be the make or break
to me, but even that is not 100% ... Can you present your work? Can you
present your work in front of your supervisor and other non-peer exec-staff
who are going to give you no room for mistakes? Can you present confidently
without arrogance? Can you support your designs with statements that speak
to a wide range of stakeholders? Can you support your design based on design
theory?

To me, presentation skills are the ones that most hiring managers don't tell
you they are looking for and few up and coming designers are trained to deal
with.

Now, here is one that might spark some volleys:
A good designer understands "the studio". Design is non-linear. If you do
not understand that to go forward you have to go sideways and backwards and
circle in place before moving towards conclusion I don't think you will be a
good designer. You will not do anything different from what a standard Bus.
Analyst (they are good people, just not designers) could do on their own and
thus you will put the design dept. in a defensive position. Please see
presentation skills again, b/c if you don't present your designs over time
w/o fear, you risk being perceived as a "black box" where stuff goes in and
other stuff goes out and people end up only looking for "pretty" in those
scenarios.

Studio, also means being able to take criticism and work with it. Design is
arguably a team sport. That doesn't in my mind contradict the model of a
strong "visionary" that leads design, but a strong visionary knows how to
take critiques and changes in requirements and re-articulate their vision
with that new criteria.

Now, the real question is how do you find all this out during an evaluation
(interview) process? To me this is where the portfolio comes in. I ask
people to present their portfolios to me as if they were presenting them to
the business, tech, and design leads for the project. I look for all the
criteria of the above during that presentation and try to give the person 3
tries to do it, especially if the first or second tries aren't panning out
as I would want them.

Here's the tough part. Out of an application pool that make it through the
door, here is what I think the statistics are in my experience for finding a
GOOD designer:

100 resumes
80%-90% don't make cut - stereotypes on purpose (asbestos is on, so
flame away)
visual designer - too aesthetic
usability expert - don't know how to speak in terms of
design, or create
HCI person - too cognitive, and don't understand aesthetics
Information architect - don't understand how to interact
over time
multimedia designer - great for games and animation, but not
for software
web designer - developer in designer's clothing

20-10% get to interview
19-9% not good
IA in designer's clothing
Never learned how to present designs
They can talk a good talk, but can't really
walk the walk
1% get the job
get it! 1 out of 100 resumes really are a GOOD IxD. Harsh
but true.

So, it is indeed REALLY hard to find a good IxD, so maybe it is so hard that
you might as well just get that eye-tracking done. ;)

-- dave

Comments

7 Jul 2006 - 12:27pm
Todd Warfel
2003

This was awesome.

On Jul 7, 2006, at 11:59 AM, David (Heller) Malouf wrote:

> 100 resumes
> 80%-90% don't make cut - stereotypes on purpose (asbestos is on, so
> flame away)
> visual designer - too aesthetic
> usability expert - don't know how to speak in terms of
> design, or create
> HCI person - too cognitive, and don't understand aesthetics
> Information architect - don't understand how to interact
> over time
> multimedia designer - great for games and animation, but not
> for software
> web designer - developer in designer's clothing
>
> 20-10% get to interview
> 19-9% not good
> IA in designer's clothing
> Never learned how to present designs
> They can talk a good talk, but can't really
> walk the walk
> 1% get the job
> get it! 1 out of 100 resumes really are a GOOD IxD. Harsh
> but true.
>
> So, it is indeed REALLY hard to find a good IxD, so maybe it is so
> hard that
> you might as well just get that eye-tracking done. ;)

Cheers!

Todd R. Warfel
Partner, Design & Usability Specialist
Messagefirst | designing and usability consulting
--------------------------------------
Contact Info
Voice: (607) 339-9640
Email: todd at messagefirst.com
AIM: twarfel at mac.com
Blog: http://toddwarfel.com
--------------------------------------
In theory, theory and practice are the same.
In practice, they are not.

7 Jul 2006 - 2:02pm
Michael Micheletti
2006

Wow, great insight, thank you.

At my previous company, we used to run a little test for UI designer
candidates during an interview. We'd trot out an example screen from an
existing working system - a screen that was designed by programmers in a
rush. It contained bad examples of just about everything you can
imagine. We'd give our candidate a printout of the screen and introduce
it as "something that we thought could perhaps be improved upon, would
they please give us some suggestions?" Responses ranged from knockout
clinics on application redesign to "hey it looks pretty good" (no hire).
But this gave us an opportunity to interact with the candidate in a
design task that only took 15 minutes. Plus it was fun.

One last thought on the resume breakdown below - I probably wouldn't
make your cut. You'd be trying to decide whether to put me in the visual
or web designer out piles. I suspect there are a fair number of other
IxD practitioners out there who shade towards one domain boundary or
another, either in history or skill set, who got to this part of your
message and said "ouch".

Michael Micheletti
Seattle, WA

-----Original Message-----
From: discuss-bounces at lists.interactiondesigners.com
[mailto:discuss-bounces at lists.interactiondesigners.com] On Behalf Of
David (Heller) Malouf
Sent: Friday, July 07, 2006 9:00 AM
To: discuss at interactiondesigners.com
Subject: [IxDA Discuss] How to hire a good IxD (RE: Eye tracking,how
valuable is it?)

[Please voluntarily trim replies to include only relevant quoted
material.]

> Care to outline exactly how one goes about hiring a *good* UI
> designer? ;-)

...

Here's the tough part. Out of an application pool that make it through
the door, here is what I think the statistics are in my experience for
finding a GOOD designer:

100 resumes
80%-90% don't make cut - stereotypes on purpose (asbestos is on,
so flame away)
visual designer - too aesthetic
usability expert - don't know how to speak in terms of
design, or create
HCI person - too cognitive, and don't understand
aesthetics
Information architect - don't understand how to interact
over time
multimedia designer - great for games and animation, but
not for software
web designer - developer in designer's clothing

20-10% get to interview
19-9% not good
IA in designer's clothing
Never learned how to present designs
They can talk a good talk, but can't
really walk the walk
1% get the job
get it! 1 out of 100 resumes really are a GOOD IxD.
Harsh but true.

So, it is indeed REALLY hard to find a good IxD, so maybe it is so hard
that you might as well just get that eye-tracking done. ;)

-- dave

________________________________________________________________
Welcome to the Interaction Design Association (IxDA)!
To post to this list ....... discuss at ixda.org List Guidelines
............ http://listguide.ixda.org/ List Help ..................
http://listhelp.ixda.org/ (Un)Subscription Options ...
http://subscription-options.ixda.org/
Announcements List ......... http://subscribe-announce.ixda.org/
Questions .................. lists at ixda.org Home .......................
http://ixda.org/ Resource Library ........... http://resources.ixda.org

7 Jul 2006 - 3:11pm
Christopher Fahey
2005

Michael Micheletti wrote:
> We'd trot out an
> example screen from an existing working system - a screen
> that was designed by programmers in a rush. It contained bad
> examples of just about everything you can imagine.

I would love to see that screen! Do you still have it?

-Cf

Christopher Fahey
____________________________
Behavior
http://www.behaviordesign.com
me: http://www.graphpaper.com

7 Jul 2006 - 4:58pm
Jared M. Spool
2003

At 11:59 AM 7/7/2006, David (Heller) Malouf wrote:
> 1% get the job
> get it! 1 out of 100 resumes really are a GOOD IxD. Harsh
>but true.

Isn't that just Sturgeon's Revelation? (90% of everything is crap. <
http://en.wikipedia.org/wiki/Sturgeon's_law >)

In this case, it's 99%.

Jared

7 Jul 2006 - 5:31pm
damon at vanves...
2006

That would make a nice headline for Nielsen's Alertbox:

"Interaction Designers 99% crap"

;) Damon

Quoting "Jared M. Spool" <jspool at uie.com>:

> Isn't that just Sturgeon's Revelation? (90% of everything is crap. <
> http://en.wikipedia.org/wiki/Sturgeon's_law >)
>
> In this case, it's 99%.
>
> Jared

9 Jul 2006 - 6:03pm
Christopher Fahey
2005

> David (Heller) Malouf wrote:
> 1% get the job
> get it! 1 out of 100 resumes really are a GOOD IxD.
> Harsh but true.

This is, perhaps, where eyetracking tools might actually be useful: If none
of the UI designers working in your organization are in that top 1%, or even
in the top 50%, then an eyetracking tool or consulting firm may well produce
better design insights than your designers ever will. The other 99 of 100
resumes eventually get jobs *somewhere*, perhaps those are the companies
that need to put their trust in an eyetracker. For my money, I'll go with a
top 1% designer for my design insights.

-Cf

Christopher Fahey
____________________________
Behavior
http://www.behaviordesign.com
212.532.4002 x203
646.338.4002 mobile

Syndicate content Get the feed