The Critique - your rules for a productive session

22 Feb 2008 - 4:42pm
6 years ago
10 replies
2690 reads
ldebett
2004

To distract you all a moment from the current frenzy of posts, I am looking
for your input on a very specific topic: the Critique.

The Critique is an art in and of itself. It is an opportunity for designers
to get feedback on their work, to illicit suggestions and to uncover areas
that need refinement or alternatives. However, not all of those who attend
critiques - or who are considered designers (IxD, UI, UE, UX, whatever) -
these days have had formal art or design training, and who therefore have
not been taught the fine art of how to effectively critique or how to be
critiqued. Currently, I have a working set of guidelines that I go by from
my education and experience, but I'm looking to hear from you all and learn
from your experiences running successful critiques.

If you wouldn't mind, during a critique of a designer's work, what are your
tips/guidelines/rules (do's and don'ts) for:

The audience members:

The designer:

(I considered seeding these to give them a start, but changed my mind to
leave it open and see what I got back instead.)

Cheers,
Lisa

Comments

22 Feb 2008 - 5:06pm
Jeff Howard
2004

My background is in graphic design and I taught it for a while so
these are a little slanted toward that discipline. Three guidelines I
learned as an undergrad and that I passed on to my students:

1. Never use phrases such as: "I like [blank]." Too subjective.
Instead frame it as "[Such and such] works because [reason]." Focus
on the principles.

2. No questions. They're a wishy washy attempt to get around making
a judgement. So instead of "I'm wondering why you used...
orange..." instead focus on the function of the design element and
address its success in that role.

3. Never apologize. This applies in practically all aspects of public
speaking. Present your work with confidence, no matter its failings,
real or imagined. Never point out the faults. People may not
recognize them, but either way they don't want to hear your excuses.
Why are you presenting bad work?

// jeff

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=26380

22 Feb 2008 - 5:45pm
Jack L. Moffett
2005

> The audience members:

Find something you think has been done well before you start
criticizing weak points.
Realize that you should be learning from others successes and
failures, not just giving feedback.

> The designer:

Give thoughtful consideration to every comment given and then
respond, rather than ripping off a gut reaction or dismissing a
comment.
Don't wait for other people to tell you what they think. Have a list
of things you want to get feedback on and ask them.

Both:
A critique is a collaboration in which everyone strives to make
things better.

Jack L. Moffett
Interaction Designer
inmedius
412.459.0310 x219
http://www.inmedius.com

In our society,
the scarce factor is not information,
it is time to attend to information.

- Herb Simon

22 Feb 2008 - 7:05pm
Chauncey Wilson
2007

Hi Lisa,

Very good question. Some thoughts.

1. What is the goal of the critique? Are you using it to teach
people about design (I've always felt that user interface inspections
are a way to do some indirect training)? Are you using it to examine
competing designs? Are you using to examine specific objects or the
overall flow?
2. In inspections (the term I'm more familiar with), there should be
an explicit way to record/track/address issues that are tangential to
the primary purpose with a clear rule about how tangential rules are
dealt with.
3. Strong comments are welcome ("there is a serious gap here that
needs some major attention") but offensive comments are not ("this
just sucks").
4. Chose people who you respect and who don't always agree with you.
Have reasonable diversity.
5. Be explicit about what kind of feedback you want. If you are
doing this early in design, you may not want feedback that objects in
a flow are a bit out of alignment.
6. Prepare some background information about who will be using the
design and make that explicit. You may get feedback that you are
missing a particular category of person.
7. Provide (brief) rationale for your comments. Like Jeff Howard,
saying that you like or dislike something is not useful. Does the
design violate a human factors principle, an aesthetic principle, or a
interaction principle. There is research around that validates the
importance of even a few words of rationale when doing inspections or
critiques.
8. The members of the audience should be told to keep their comments
brief and not tell war stories which waste time and often do not add
much (if it is a really good story, tell it later).
9. If there is a global problem (one that occurs in multiple places)
consider a guideline about how to address that. The same problem or
flaw might be manifested in a slightly different way so you might
bring that up but if the design flaw is the same on multiple
artifacts, then you might just note that this is a global problem and
not bring it up.
10. Note conflicts -- one person states that something is a delightful
elements because.... while someone else feels that it is a major
flaw because.... Ask a question if you get serious diametrically
opposed feedback. You might find that there is a bimodal distribution
that would yield different feedback.
11. Some articles I've read state that the designer who is being
critiqued should run the session; others recommend a facilitator. good
arguments for both.

Good topic to discuss at The Asgard next week.

Chauncey

On Fri, Feb 22, 2008 at 4:42 PM, Lisa deBettencourt <ldebett at gmail.com> wrote:
> To distract you all a moment from the current frenzy of posts, I am looking
> for your input on a very specific topic: the Critique.
>
> The Critique is an art in and of itself. It is an opportunity for designers
> to get feedback on their work, to illicit suggestions and to uncover areas
> that need refinement or alternatives. However, not all of those who attend
> critiques - or who are considered designers (IxD, UI, UE, UX, whatever) -
> these days have had formal art or design training, and who therefore have
> not been taught the fine art of how to effectively critique or how to be
> critiqued. Currently, I have a working set of guidelines that I go by from
> my education and experience, but I'm looking to hear from you all and learn
> from your experiences running successful critiques.
>
> If you wouldn't mind, during a critique of a designer's work, what are your
> tips/guidelines/rules (do's and don'ts) for:
>
>
> The audience members:
>
>
> The designer:
>
>
>
> (I considered seeding these to give them a start, but changed my mind to
> leave it open and see what I got back instead.)
>
> Cheers,
> Lisa
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help
>

22 Feb 2008 - 8:16pm
White, Jeff
2007

Awesome topic!

This is so relevant to the work I'm doing now, and I'm happy to see
others bringing this up for discussion. My input is in the context of
a bunch of people bringing their work into a room, presenting it, then
the whole room critiquing each idea. Not really a stakeholder review
of a prototype, participatory design session, JAD session, etc.

Here are the ground rules that have allowed my design team to run
successful critique sessions with developers (yes! eek! those evil
developers who most of us seem to think should just wait on the other
side of the wall for our design specs to come flying over and then
implement our grand visions with no questions asked!) yes, I know I'm
exaggerating. It's Friday :-) Really though - letting the developers
we work with participate in concept ideation sketches and go through
critique has been a huge win for our organization in lots of different
ways. There are lots of interesting relationships between this
approach and the things we heard our thought leaders speak about at
interaction08. Or if you didn't attend, the things our thought leaders
spoke about at interaction08. :-)

A colleague of mine and I have been conducting "design studios" with
teams of engineers over the past year. It has been a huge success for
reasons I won't drone on about right now because it's a bit off topic.
But here are the ground rules we establish with our teams:

1. Criticism is a gift that you need to learn how to embrace and be
thankful for. Letting your work be critiqued, critiquing your own
work, and critiquing the work of others builds your design chops and
communication skills in many different ways. The critique process is
key for any designer, no matter what the discipline - graphic design,
interaction design, industrial design, architecture, etc.

2. Critique the design, not the designer. It's not a personal attack.
Be professional.

3. Others finding flaws in your work is a good thing, it doesn't mean
you did a bad job, it just means the critique process is working.
Again, don't take it personally, learn to use it to make your future
design efforts all the better. Spotting flaws early on (especially if
you're in the sketch phase, before any code is present) is priceless
for any project.

4. There's some good in every design. One germ of goodness in a
"horrid" design can turn into a huge, positive idea.

That's pretty much it. Lots more to say, but it all stems from those
core principles.

Thanks for a fun topic,

Jeff

On Fri, Feb 22, 2008 at 7:05 PM, Chauncey Wilson
<chauncey.wilson at gmail.com> wrote:
> Hi Lisa,
>
> Very good question. Some thoughts.
>
> 1. What is the goal of the critique? Are you using it to teach
> people about design (I've always felt that user interface inspections
> are a way to do some indirect training)? Are you using it to examine
> competing designs? Are you using to examine specific objects or the
> overall flow?
> 2. In inspections (the term I'm more familiar with), there should be
> an explicit way to record/track/address issues that are tangential to
> the primary purpose with a clear rule about how tangential rules are
> dealt with.
> 3. Strong comments are welcome ("there is a serious gap here that
> needs some major attention") but offensive comments are not ("this
> just sucks").
> 4. Chose people who you respect and who don't always agree with you.
> Have reasonable diversity.
> 5. Be explicit about what kind of feedback you want. If you are
> doing this early in design, you may not want feedback that objects in
> a flow are a bit out of alignment.
> 6. Prepare some background information about who will be using the
> design and make that explicit. You may get feedback that you are
> missing a particular category of person.
> 7. Provide (brief) rationale for your comments. Like Jeff Howard,
> saying that you like or dislike something is not useful. Does the
> design violate a human factors principle, an aesthetic principle, or a
> interaction principle. There is research around that validates the
> importance of even a few words of rationale when doing inspections or
> critiques.
> 8. The members of the audience should be told to keep their comments
> brief and not tell war stories which waste time and often do not add
> much (if it is a really good story, tell it later).
> 9. If there is a global problem (one that occurs in multiple places)
> consider a guideline about how to address that. The same problem or
> flaw might be manifested in a slightly different way so you might
> bring that up but if the design flaw is the same on multiple
> artifacts, then you might just note that this is a global problem and
> not bring it up.
> 10. Note conflicts -- one person states that something is a delightful
> elements because.... while someone else feels that it is a major
> flaw because.... Ask a question if you get serious diametrically
> opposed feedback. You might find that there is a bimodal distribution
> that would yield different feedback.
> 11. Some articles I've read state that the designer who is being
> critiqued should run the session; others recommend a facilitator. good
> arguments for both.
>
> Good topic to discuss at The Asgard next week.
>
> Chauncey
>
>
>
> On Fri, Feb 22, 2008 at 4:42 PM, Lisa deBettencourt <ldebett at gmail.com> wrote:
> > To distract you all a moment from the current frenzy of posts, I am looking
> > for your input on a very specific topic: the Critique.
> >
> > The Critique is an art in and of itself. It is an opportunity for designers
> > to get feedback on their work, to illicit suggestions and to uncover areas
> > that need refinement or alternatives. However, not all of those who attend
> > critiques - or who are considered designers (IxD, UI, UE, UX, whatever) -
> > these days have had formal art or design training, and who therefore have
> > not been taught the fine art of how to effectively critique or how to be
> > critiqued. Currently, I have a working set of guidelines that I go by from
> > my education and experience, but I'm looking to hear from you all and learn
> > from your experiences running successful critiques.
> >
> > If you wouldn't mind, during a critique of a designer's work, what are your
> > tips/guidelines/rules (do's and don'ts) for:
> >
> >
> > The audience members:
> >
> >
> > The designer:
> >
> >
> >
> > (I considered seeding these to give them a start, but changed my mind to
> > leave it open and see what I got back instead.)
> >
> > Cheers,
> > Lisa
> > ________________________________________________________________
> > Welcome to the Interaction Design Association (IxDA)!
> > To post to this list ....... discuss at ixda.org
> > Unsubscribe ................ http://www.ixda.org/unsubscribe
> > List Guidelines ............ http://www.ixda.org/guidelines
> > List Help .................. http://www.ixda.org/help
> >
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help
>

25 Feb 2008 - 2:55am
Jonas Löwgren
2003

Lisa,

From my position as design teacher, it is unusual to speak about
"critique" involving only an audience and a designer -- but not a
teacher.

I suppose I would rather call this a review or an inspection, saving
the term critique for pedagogical settings where a learning designer
is presenting work for a teacher and a group of fellow students, and
the whole session has the primary intentions that the participating
students should learn.

I can think of guidelines for the pedagogical critique session, of
course, and some of them overlap with what others have already
mentioned.

But I will not go further into the pedagogical situation as I gather
that you are more interested in the "productive critique session"
where the main goal is to improve the quality of the design work.

Regards,

Jonas Löwgren

25 Feb 2008 - 9:56am
Benjamin Ho
2007

I too do not use the word critique as that's more intended for
architecture students and their peers to bash at one-another. ;)
(But I digress.)

Instead, there are a few components in these Design Reviews:

1. Heuristic evaluation - done usually by the usability analyst;
2. Findings evaluation - design changes and suggestions brought on
by user testing. Anyone can make these suggestions so long as they
review the data;
3. General feedback - either by the users, development and sales
which are always taken into consideration but must be weighed-in to
its validity. There are still principles to follow.

While I do both design and testing, I keep objective by using the
data that I compile and make changes wherever necessary.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=26380

25 Feb 2008 - 12:16pm
Jack L. Moffett
2005

On Feb 25, 2008, at 2:55 AM, Jonas Löwgren wrote:

> But I will not go further into the pedagogical situation as I gather
> that you are more interested in the "productive critique session"
> where the main goal is to improve the quality of the design work.

Jonas,

The critique, as practiced in design schools, is applicable to
projects at a professional level too.

Jack

Jack L. Moffett
Interaction Designer
inmedius
412.459.0310 x219
http://www.inmedius.com

Design is like California.
No one is born there.

-Dick Buchanan

25 Feb 2008 - 1:08pm
Jonas Löwgren
2003

> The critique, as practiced in design schools, is applicable to
> projects at a professional level too.

Jack,

I guess that was the thought of the original poster.

However, one of the first replies provided a guideline in the general
direction of "never ask questions of the presenter, only state
grounded assessments."

When I read that, I realized that the contexts must be quite different.

One of my essential techniques for *teaching* during a critique
session is to combine questions with my own assessments.

The ideal is to create a pedagogical situation where the presenting
student and I together construct a rationale (or a line of reasoning)
on the fly which

1. supports some of the design decisions,
2. questions other design decisions,
3. hints at promising alternatives, and
4. directs the attention of the presenting student (and the class) to
other noteworthy work in the domain.

When it works, it is great.
When it doesn't work, it is awkward and sometimes even embarassing.

But I tend to stick to it, as I think the pedagogical benefits
outweigh the inconveniences. However, I am certainly not claiming
that it is a good technique if the aim is strictly to improve the
product. Hence my attempt to distinguish "pedagogical critique
sessions" from "constructive critique sessions".

Regards,
Jonas Löwgren

25 Feb 2008 - 2:08pm
Jack L. Moffett
2005

On Feb 25, 2008, at 1:08 PM, Jonas Löwgren wrote:

> Hence my attempt to distinguish "pedagogical critique sessions"
> from "constructive critique sessions".

Gotcha. That makes sense.

Jack

Jack L. Moffett
Interaction Designer
inmedius
412.459.0310 x219
http://www.inmedius.com

The details are not the details.
They make the design.

-Charles Eames

26 Feb 2008 - 7:16pm
ldebett
2004

These are all great points and I appreciate you taking the time to share
them with (us) me.

Yes, I called it a "critique" because, well, when you learn certain terms in
school, you tend to bring them with you to your day job. ;-) If I could to a
mediocre job summarizing my own experience, I'd say the goals of design
reviews have typically been twofold and very symbiotic, kind of like a
dance:

1. the designer informs others of the state of the design: kind of a "if we
were to ship it now, here's what it would look/act/be like", and explains (I
hate saying "defends" because it shouldn't be an attack) their rationale,
intent, missing pieces, next steps, etc.
2. the audience weighs in on it from their perspectives; either making sure
the design meets their needs (quantitative) or as a peer making suggestions
for improvement (qualitative), and encourages focus for further
investigation in the right areas.

The subtle nuances of the dance can either make the session successful and
move the design forward in leaps and bounds or, well, not.

Since Jonas discussed the professor/teacher's role, has anyone had success
in (or thoughts about) having a third person be a "moderator" in
professional design reviews such as a manager?

A few rules I've worked by:

Designer: Don't take things personally. It's about the design, not you. And
you are not your design. And if someone's being a real jerk, either get them
to explain themselves more or ask others for their thoughts on that person's
comment. There could be a nugget of gold in there.

Audience: Try to understand the design and look for both positives and
negatives. Point out the things that do work so they don't get deleted in
trying to fix the things that don't. And don't be that jerk. What you say is
as important as how you say it. And back things up with solid reasoning.
Saying "I don't like it" vs. "the design of that component gives a false
affordance..." produces very different results.

~Lisa

Syndicate content Get the feed