interview "test" ideas for Usability position?

23 Aug 2010 - 7:59pm
4 years ago
6 replies
998 reads
scottymac
2010

Hi, I'm trying to come up with a "test" to be used as part of the interview process for a Usability (web) position - so I thought I'd ask the community for any ideas, nuggets, spitballs to get our brains going...

I work for a smallish web team within a large organization and we are looking to hire someone inhouse for the Usability side of things. The role is mainly Usability analysis, user testing, best practices, recommendations, etc for our exisiting large content web sites and new areas/ideas for the sites as well.

Part of the formal interview process has to have a "test" - give the interviewee a problem or task to solve in 20 minutes and present their solution at the end. The point is to see how someone thinks. We try and make these fun - relivant to the position, but not totally expected or super serious.

We're kind of at a blank for the Usability position. Any ideas? Something you've come across in the past? Something you wish you could have done or asked in an interview process?

Many thanks in advance!

Comments

23 Aug 2010 - 9:22pm
Mathew Sanders
2009

When involved in recruitment I've always liked to ask people to describe the design process they go through from from beginning to end, it's amazing how many people we interviewed don't even think about understanding a design problem or goal - for them design is just a surface veneer.

From a best practice perspective you could give people a screen shot of a high use page that you've previously designed and give them 10 minutes to create a list of what they see as good and bad aspects of the design.

You could also give them a specific problem (e.g. people on an e-commerce site are browsing products but not adding to the cart), give them a hypothetical budget (e.g. 5 days) and ask them to describe what they would do - I think adding a time/budget constraint would be good because it also allows you to judge how people prioritise in less than idea situations.

Of course, in 20 minutes you're probably not going to get practical advice, so don't look for accuracy/correctness of responses, but instead signs that people have the right technique - e.g. asking questions, focusing on the problem.

Hope that gives you some ideas, will be interesting to see what other people suggest as well :)

23 Aug 2010 - 10:05pm
dgavales
2010

Hi,

I had a very engaging experience when being interviewed for a user research position at a large software company. I was told to assume I had x amount of time to develop a new appliance (which the interviewer specified). I was then asked to walk through how I would start designing this appliance, thinking aloud what my process would be. As I raised considerations and shared what questions I would be asking, the interviewer would supply bits of information. As I learned more about the intended users and business constraints, I was asked how I would find out more about user needs, respond to setbacks, prioritize issues and activities, etc.

Although we didn't discuss software at all, it was a great way for them to learn how I think about the design process and usability's role in it, as well as the methods I had used.

Diana

> >

24 Aug 2010 - 5:52am
smitty777
2010

I've given interviews similar to the ones Matthew describes.  At one company, we sent out a link to a site we had re-designed a few days before the interview and had them send in a one page "report".  I don't know if they're still doing it, but I know the Cooper web site had something similar in their Careers section.  They presented you with some type of MS Calendar function (IIRC), and asked you how you would improve it.

I actually like Matthew's technique better, as it allows you to see how they think on their toes, and prevents them from enlisting outside help.  

24 Aug 2010 - 6:05am
Chauncey Wilson
2007

Hi,   There are seveal things you could do to evaluate a person for a usability position.   1.  Have the person do an interview.  Much of usability work is asking good questions of usability participants and stakeholders.  I've been in the field since the 1970s and have been amazed that in dozens of job interviews over those years, no one asked me to do an interview or to facilitate a session. 

2.  Ask the person to write up a short test plan about how to test given a few constraints.  They have 30 minutes and then have to present.  In this evaluation, you could have a checklist of things that you are looking for (for consistency and fairness).  For example, does the person list "sampling method" (a critical factor in any evaluation) and business goals, and data analysis methods.

3.  Analyzing qualitative data is something that every usability practitioner must do.  What is you have a sample of data and ask the person to organize it and extract a small set of conclusions. 4.  Ask the person to create an anti-resume related to usability that describes what he/she is not good at, has not done, has not used, etc.  5.  Ask the person to discuss ethical issues around usability practices.  6.  Usability is a very general term so you might ask them to describe what "usability" means and discuss how usability attributes of most concern can be different (for example, products that are only used once should focus on initial learnability, while products that people use all the time might focus on efficiency.  Ask the person to define usability in a quantitative sense (there is much in the literature on usability goals).

7.  Ask the person how he/she would track their effectiveness over the course of several years and how they would justify their work.  This is a hard question for anyone, but is one that many of us have to answser to.

8.  This might be more of a design issue, but you could ask a person to discuss all the usability/design issues with a simple object like a "list".  I did a list deconstruction with some colleagues and came up with more than 30 issues including:  international issues, ordering, selection, focus issue, knowing how many are selected in a long list, location of activation buttons, feedback, length of the list, whether it is dynamic or static, filtering or not...... 

9.  Listening is important so you might design a listening exercise.  10.  Have the person review a 10-minute bit of video tape and the reflect on the moderation, the interaction with the participant, the problems that emerged, etc. 11.  Ask the person to watch a section, take notes, and then present a 5 minute summary of what he/she saw. 12.  Provide some dilemmas and ask them how they might go about and solve them. 13.  Ask them how they would introduce usability to a development team - perhaps they could make a short list of topics.   Chauncey

On Mon, Aug 23, 2010 at 9:29 PM, scottymac <meousings@gmail.com> wrote:

Hi, I'm trying to come up with a "test" to be used as part of the interview process for a Usability (web) position - so I thought I'd ask the community for any ideas, nuggets, spitballs to get our brains going...

I work for a smallish web team within a large organization and we are looking to hire someone inhouse for the Usability side of things. The role is mainly Usability analysis, user testing, best practices, recommendations, etc for our exisiting large content web sites and new areas/ideas for the sites as well.

Part of the formal interview process has to have a "test" - give the interviewee a problem or task to solve in 20 minutes and present their solution at the end. The point is to see how someone thinks. We try and make these fun - relivant to the position, but not totally expected or super serious.

We're kind of at a blank for the Usability position. Any ideas? Something you've come across in the past? Something you wish you could have done or asked in an interview process?

Many thanks in advance!

(((Please leave all content bel
24 Aug 2010 - 7:06am
Brian Sullivan
2009

To piggyback off of Chauncey's remarks, you can:

  1. Have the person perform a heuristic evaluation for 30 minutes. (Judge thier knowledge)
  2. Then, have the person present very casually to the team their findings. (Judge the presentation style)
  3. Show a design of a bad user interface. Have the person fix it on a white board via sketching (show analysis and recommendation)

Thanks, Brian Sullivan

From: ixdaor@host.ixda.org on behalf of Chauncey Wilson Sent: Tue 8/24/2010 6:36 AM To: Sullivan, Brian Subject: Re: [IxDA] interview "test" ideas for Usability position?

Hi,

There are seveal things you could do to evaluate a person for a usability position.

  1. Have the person do an interview. Much of usability work is asking good questions of usability participants and stakeholders. I've been in the field since the 1970s and have been amazed that in dozens of job interviews over those years, no one asked me to do an interview or to facilitate a session.

  2. Ask the person to write up a short test plan about how to test given a few constraints. They have 30 minutes and then have to present. In this evaluation, you could have a checklist of things that you are looking for (for consistency and fairness). For example, does the person list "sampling method" (a critical factor in any evaluation) and business goals, and data analysis methods.

  3. Analyzing qualitative data is something that every usability practitioner must do. What is you have a sample of data and ask the person to organize it and extract a small set of conclusions.

  4. Ask the person to create an anti-resume related to usability that describes what he/she is not good at, has not done, has not used, etc.
  5. Ask the person to discuss ethical issues around usability practices.
  6. Usability is a very general term so you might ask them to describe what "usability" means and discuss how usability attributes of most concern can be different (for example, products that are only used once should focus on initial learnability, while products that people use all the time might focus on efficiency. Ask the person to define usability in a quantitative sense (there is much in the literature on usability goals).

  7. Ask the person how he/she would track their effectiveness over the course of several years and how they would justify their work. This is a hard question for anyone, but is one that many of us have to answser to.

  8. This might be more of a design issue, but you could ask a person to discuss all the usability/design issues with a simple object like a "list".
    I did a list deconstruction with some colleagues and came up with more than 30 issues including: international issues, ordering, selection, focus issue, knowing how many are selected in a long list, location of activation buttons, feedback, length of the list, whether it is dynamic or static, filtering or not......

  9. Listening is important so you might design a listening exercise.

  10. Have the person review a 10-minute bit of video tape and the reflect on the moderation, the interaction with the participant, the problems that emerged, etc.
  11. Ask the person to watch a section, take notes, and then present a 5 minute summary of what he/she saw.
  12. Provide some dilemmas and ask them how they might go about and solve them.
  13. Ask them how they would introduce usability to a development team - perhaps they could make a short list of topics.

Chauncey

On Mon, Aug 23, 2010 at 9:29 PM, scottymac wrote:

>Hi, I'm trying to come up with a "test" to be used as part of the interview >process for a Usability (web) position - so I thought I'd ask the community >for any ideas, nuggets, spitballs to get our brains going... > >I work for a smallish web team within a large organization and we are >looking to hire someone inhouse for the Usability side of things. The role >is mainly Usability analysis, user testing, best practices, recommendations, >etc for our exisiting large content web sites and new areas/ideas for the >sites as well. > >Part of the formal interview process has to have a "test" - give the >interviewee a problem or task to solve in 20 minutes and present their >solution at the end. The point is to see how someone thinks. We try and make >these fun - relivant to the position, but not totally expected or super >serious. > >We're kind of at a blank for the Usability position. Any ideas? Something >you've come across in the past? Something you wish you could have done or >asked in an interview process? > >Many thanks in advance! > >(((Please leave all content bel >

24 Aug 2010 - 11:05am
Diana Wynne
2008

Most of the responses have focused on the usability analysis part of the job. I agree it's important to know if someone has these skills to do a heuristic evaluation and make practical recommendations. 
But in an interview, I'd suggest you focus on the things that will make this person effective: the ability to navigate contradictions and conflicts that go with this role.
Users often say something is easy when they're struggling. The designer is one of the company founders, and doesn't respond well to "criticism."Testing surfaces a serious issue that there isn't time to address in the current schedule.The engineering team is often reluctant to rebuild work they've already developed. Marketing likes the idea of user testing, but really doesn't want to hear about problems, only happy customers.etc.
It's much less important to know how your candidate would redesign the page/app in an ideal world if you already have designers. Versus understanding how she would advocate for users in your real world process.
Hope this is helpful.Diana

On Mon, Aug 23, 2010 at 6:16 PM, scottymac <meousings@gmail.com> wrote:

Hi, I'm trying to come up with a "test" to be used as part of the interview process for a Usability (web) position - so I thought I'd ask the community for any ideas, nuggets, spitballs to get our brains going...

I work for a smallish web team within a large organization and we are looking to hire someone inhouse for the Usability side of things. The role is mainly Usability analysis, user testing, best practices, recommendations, etc for our exisiting large content web sites and new areas/ideas for the sites as well.

Part of the formal interview process has to have a "test" - give the interviewee a problem or task to solve in 20 minutes and present their solution at the end. The point is to see how someone thinks. We try and make these fun - relivant to the position, but not totally expected or super serious.

We're kind of at a blank for the Usability position. Any ideas? Something you've come across in the past? Something you wish you could have done or asked in an interview process?

Many thanks in advance!

(
Syndicate content Get the feed