How many alternatives, concepts, or sketches are enough?

19 Jan 2009 - 5:13am
5 years ago
11 replies
1711 reads
Jakub Linowski
2008

http://www.90percentofeverything.com/2009/01/14/why-you-shouldnt-rush-into-a-solution-too-quickly/

Harry posted an interesting post on "90 percent of everything" about not
rushing to design solutions too quickly. Designers should cover the design
space with divergent approaches first and identify proper alternatives
before converging on an idea. I think I've heard others say as well the same
about iterative design, and the ability of successful designs only to evolve
if the pool of ideas is rich and diverse. The idea is not exactly
revolutionary but stirs a basic design question.

The question which I am wondering about then is how do we know how many
alternatives are enough? How do we know we have enough sketches,
alternatives or concepts before we begin choosing a satisfising solution.

Harry also pointed me to wiki entry on wicked problems
http://en.wikipedia.org/wiki/Wicked_problem, where it says that it's not
possible to measure the design space. "Wicked problems do not have an
enumerable (or an exhaustively describable) set of potential solutions, nor
is there a well-described set of permissible operations that may be
incorporated into the plan." So what are we left with here?

Comments

19 Jan 2009 - 6:00am
Jonas Löwgren
2003

I guess design sometimes deals with problems that are wicked in
Rittel's original sense of the word, and sometimes not.

A related way to think, which has proven more generative to me in
terms of process management, is to say that design is about learning.

What you do as a designer, and particularly in early, explorative
phases, is to learn as much as possible. You want to learn about what
the design space looks like, how the "problem" can be framed in
different ways (sometimes equal to different kinds of transformations
from an existing situation), what possible "solutions" there might
be, what qualities you might expect from those "solutions" if they
were deployed.

This kind of learning is normally not limited by inherent bounds of
the design space. There is always another idea that could be
explored, always another way to rephrase the "problem".

Hence, in my experience, you do not work broadly and divergently in
order to increase your certainty as much as in order to reduce your
uncertainty.

And the question of when we have enough is often answered by other
means, such as when time and resources devoted to exploration are
exhausted. At that point, we have to obey the 80/20 rule and hope for
good-enough.

Sorry if this comes across as too academic and Zen-like, but this is
actually how I tend to think about my work and my teaching.

Jonas Löwgren

19 Jan 2009 - 7:31am
Dave Malouf
2005

I take the pop-corn in the microwave approach to this. take it out
when the pops start to happen infrequently. But as Jonas says usually
other factors create limitations before this.

BTW, sketching/exploration, is not to create "alternatives" and
"iterations" but is a ideation generation process. Even though you
do 100 sketches, only 10 concrete ideas may come out of the process.

-- dave

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=37356

19 Jan 2009 - 11:37am
Dana Chisnell
2008

I recommend that you test your alternate designs by having users use
prototypes of them.

Teams I've worked with have processes for developing radically
different alternatives or approaches to design problems. Then they
have people who are users, or like the real users, try out one of the
alternatives.

Here's how it works: Say your team decides on two design ideas
(testing more than that at at time is actually really hard) that are
really different. Now, find a bunch of users maybe 10 or 12 (but at
least 8). Separate them into two groups. Create task scenarios that
you want users to follow in trying out the prototypes. Everyone does
the same tasks. Now, individually, have each person in one group do
the tasks on prototype A. Then have each person in the other group do
the tasks on prototype B. The whole team observes all the sessions,
watching for where users have more or fewer problems.

My experience is that parts of each design work well and parts of each
design work poorly, but out of this data you get from users, you can
create a hybrid design that should work pretty well. (And which, by
the way, you should have a few more users try out with the same tasks.)

The beauty of this approach is that it frees the designers to use
their creativity and design knowledge, and then you can measure how
effective those creative ideas are by having users use them. It's no
longer a subjective decision among the team members.

Make sense?

Dana

:: :: :: :: :: :: :: :: :: :: :: :: :: :: :: :: :: :: :: :: :: :: :: :: :: :: ::
Dana Chisnell
desk: 415.392.0776
mobile: 415.519.1148

dana AT usabilityworks DOT net

www.usabilityworks.net
http://usabilitytestinghowto.blogspot.com/

On Jan 19, 2009, at 3:13 AM, Jakub Linowski wrote:

> http://www.90percentofeverything.com/2009/01/14/why-you-shouldnt-rush-into-a-solution-too-quickly/
>
> Harry posted an interesting post on "90 percent of everything" about
> not
> rushing to design solutions too quickly. Designers should cover the
> design
> space with divergent approaches first and identify proper alternatives
> before converging on an idea. I think I've heard others say as well
> the same
> about iterative design, and the ability of successful designs only
> to evolve
> if the pool of ideas is rich and diverse. The idea is not exactly
> revolutionary but stirs a basic design question.
>
> The question which I am wondering about then is how do we know how
> many
> alternatives are enough? How do we know we have enough sketches,
> alternatives or concepts before we begin choosing a satisfising
> solution.
>
> Harry also pointed me to wiki entry on wicked problems
> http://en.wikipedia.org/wiki/Wicked_problem, where it says that it's
> not
> possible to measure the design space. "Wicked problems do not have an
> enumerable (or an exhaustively describable) set of potential
> solutions, nor
> is there a well-described set of permissible operations that may be
> incorporated into the plan." So what are we left with here?
>
>
> ________________________________________________________________
> Reply to this thread at ixda.org
> http://www.ixda.org/discuss?post=37356
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help

19 Jan 2009 - 12:25pm
Chauncey Wilson
2007

I would be curious to hear what tools colleagues do use for prioritization
of ideas. The key issue here is what the criteria are for choosing ideas.
In the early stages of ideation, the criteria might be different for
choosing what to consider further (the 10 ideas out of 300) versus what to
consider when you move into detailed design.

Some general methods for prioritization are:

1. The monetary method where a sample of people are given a fixed amount of
"money", a list of ideas or requirements along with their relative costs and
then asked to "buy" the things of most value.
2. The criterion matrix where you list the criteria (weighted or
unweighted) and then calculate a score with the top scores meeting more of
the criteria.
3. Q-sorting where you ask people to sort on an important criteria on a
scale ranging from low to high.
4. Private voting for the best ideas
5. Public voting for the best ideas (red dots on the best ideas)
6. Consensus
7. Decision by a leader
8. Decision by another group
9. The target method (good for a first cut between good and not-good idea)

In braindrawing exercises, the design team would look at lots of sketches
and mark ideas that seem worth pursuing which would be consensus or voting
and would then have a product team do a second level of prioritization on
specific criteria.

What other techniques do people use? This is something that doesn't seem to
get discussed much.

Chauncey

On Mon, Jan 19, 2009 at 12:41 PM, christine chastain <
chastain.christine at gmail.com> wrote:

> More than the issue of "how many ideas", I always end up without adequate
> prioritization mechanisms/tools by which to decide alternatives to choose
> for inclusion in the iteration process.
>
> On Mon, Jan 19, 2009 at 7:31 AM, Dave Malouf <dave.ixd at gmail.com> wrote:
>
> > I take the pop-corn in the microwave approach to this. take it out
> > when the pops start to happen infrequently. But as Jonas says usually
> > other factors create limitations before this.
> >
> > BTW, sketching/exploration, is not to create "alternatives" and
> > "iterations" but is a ideation generation process. Even though you
> > do 100 sketches, only 10 concrete ideas may come out of the process.
> >
> > -- dave
> >
> >
> > . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> > Posted from the new ixda.org
> > http://www.ixda.org/discuss?post=37356
> >
> >
> > ________________________________________________________________
> > Welcome to the Interaction Design Association (IxDA)!
> > To post to this list ....... discuss at ixda.org
> > Unsubscribe ................ http://www.ixda.org/unsubscribe
> > List Guidelines ............ http://www.ixda.org/guidelines
> > List Help .................. http://www.ixda.org/help
> >
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help
>

19 Jan 2009 - 12:59pm
Scott Berkun
2008

All of these methods you listed strike me as limiting in they emphasize
equal voting - often I don't believe everyone deserves an equal vote.
Heretical perhaps, but I'd much rather let a small number of people who will
be held accountable for the final design entirely drive these explorations.
It's their necks on the line. They should at least win or lose on their own
intuitions.

Having people vote on one sentence, or one sketch, descriptions of ideas is
always a crap-shoot: people are heavily biased to the ideas they're familiar
with, and they can't be equally familiar with all the ideas.

With a pile of 50 ideas and only time to explore 5, I'd sit down with the
three or four people most accountable for the final result and talk it out.
I would depend on intuition, debate and persuasion more than any sort of
numerical/polling/ranking system.

If I did anything "methody", which I'd try to avoid, I do one of two things:

1) Have a list of criteria, or project goals, or desirable attributes up on
the whiteboard during that discussion to help us frame our opinions.

2) Make the goal to pick one high risk idea, three medium risk ideas, and
one low risk idea. This frames the problem of picking alternatives as a risk
portfolio, where our goal is to distribute the creative risks in some way.
This makes it ok to advocate a crazy idea, since that's desirable to fit the
high risk slot.

But most importantly, if I didn't have the power to grant this much
authority to those 3 people, my real problem is political, not the quest for
the perfect number of alternatives.

-Scott

Scott Berkun
www.scottberkun.com

-----Original Message-----
From: discuss-bounces at lists.interactiondesigners.com
[mailto:discuss-bounces at lists.interactiondesigners.com] On Behalf Of
Chauncey Wilson
Sent: Monday, January 19, 2009 10:26 AM
To: christine chastain
Cc: Dave Malouf; discuss at ixda.org
Subject: Re: [IxDA Discuss] How many alternatives, concepts,or sketches are
enough?

I would be curious to hear what tools colleagues do use for prioritization
of ideas. The key issue here is what the criteria are for choosing ideas.
In the early stages of ideation, the criteria might be different for
choosing what to consider further (the 10 ideas out of 300) versus what to
consider when you move into detailed design.

Some general methods for prioritization are:

1. The monetary method where a sample of people are given a fixed amount of
"money", a list of ideas or requirements along with their relative costs and
then asked to "buy" the things of most value.
2. The criterion matrix where you list the criteria (weighted or
unweighted) and then calculate a score with the top scores meeting more of
the criteria.
3. Q-sorting where you ask people to sort on an important criteria on a
scale ranging from low to high.
4. Private voting for the best ideas
5. Public voting for the best ideas (red dots on the best ideas) 6.
Consensus 7. Decision by a leader 8. Decision by another group 9. The
target method (good for a first cut between good and not-good idea)

19 Jan 2009 - 1:44pm
Chauncey Wilson
2007

You make a good point though I didn't specifically mention equal voting at
all. You could have a small group who, as you say, have their necks on the
line or you could have private voting of the 10 top designers in the country
using polling software or you could generate criteria and have your small
group use the criteria as a starting point for a deeper discussion of the
type you suggest. You mention listing the criteria on the board which is a
great starting point, because many groups fail to explicitly identify
criteria that they are using (that method sounds like the QOC method -
Questions-Options-Criteria - that is described in the "design rationale"
literature.)

Some time ago, I worked with a group of people who necks were on the line
and the use of a group Q-sort on the dimension of 'project risk" for
particular requirements worked much as you described with the different
items getting much discussion among respected team members and then getting
placed into low, medium, and high risks. The discussion for each item often
elaborated on what was risky for the different representatives.

Chauncey

On Mon, Jan 19, 2009 at 1:59 PM, Scott Berkun <info at scottberkun.com> wrote:

>
> All of these methods you listed strike me as limiting in they emphasize
> equal voting - often I don't believe everyone deserves an equal vote.
> Heretical perhaps, but I'd much rather let a small number of people who
> will
> be held accountable for the final design entirely drive these explorations.
> It's their necks on the line. They should at least win or lose on their own
> intuitions.
>
> Having people vote on one sentence, or one sketch, descriptions of ideas is
> always a crap-shoot: people are heavily biased to the ideas they're
> familiar
> with, and they can't be equally familiar with all the ideas.
>
> With a pile of 50 ideas and only time to explore 5, I'd sit down with the
> three or four people most accountable for the final result and talk it out.
> I would depend on intuition, debate and persuasion more than any sort of
> numerical/polling/ranking system.
>
> If I did anything "methody", which I'd try to avoid, I do one of two
> things:
>
> 1) Have a list of criteria, or project goals, or desirable attributes up on
> the whiteboard during that discussion to help us frame our opinions.
>
> 2) Make the goal to pick one high risk idea, three medium risk ideas, and
> one low risk idea. This frames the problem of picking alternatives as a
> risk
> portfolio, where our goal is to distribute the creative risks in some way.
> This makes it ok to advocate a crazy idea, since that's desirable to fit
> the
> high risk slot.
>
> But most importantly, if I didn't have the power to grant this much
> authority to those 3 people, my real problem is political, not the quest
> for
> the perfect number of alternatives.
>
> -Scott
>
> Scott Berkun
> www.scottberkun.com
>
>
> -----Original Message-----
> From: discuss-bounces at lists.interactiondesigners.com
> [mailto:discuss-bounces at lists.interactiondesigners.com] On Behalf Of
> Chauncey Wilson
> Sent: Monday, January 19, 2009 10:26 AM
> To: christine chastain
> Cc: Dave Malouf; discuss at ixda.org
> Subject: Re: [IxDA Discuss] How many alternatives, concepts,or sketches are
> enough?
>
> I would be curious to hear what tools colleagues do use for prioritization
> of ideas. The key issue here is what the criteria are for choosing ideas.
> In the early stages of ideation, the criteria might be different for
> choosing what to consider further (the 10 ideas out of 300) versus what to
> consider when you move into detailed design.
>
> Some general methods for prioritization are:
>
> 1. The monetary method where a sample of people are given a fixed amount
> of
> "money", a list of ideas or requirements along with their relative costs
> and
> then asked to "buy" the things of most value.
> 2. The criterion matrix where you list the criteria (weighted or
> unweighted) and then calculate a score with the top scores meeting more of
> the criteria.
> 3. Q-sorting where you ask people to sort on an important criteria on a
> scale ranging from low to high.
> 4. Private voting for the best ideas
> 5. Public voting for the best ideas (red dots on the best ideas) 6.
> Consensus 7. Decision by a leader 8. Decision by another group 9. The
> target method (good for a first cut between good and not-good idea)
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help
>

19 Jan 2009 - 2:01pm
Scott Berkun
2008

True - you didn't - Sorry for criticizing you for something you did not say
:)

My bias is against teams pretending to quantify the unquantifiable. I like
opinions. I like things that designers believe but can not prove
mathematically, but can explain through argument. Any decision making
process that doesn't make use of conviction, persuasion and passion is one I
doubt will work out well.

Decision models/methods are great provided they're fodder - that they're
used to help the discussion and debate, but not to replace it. Too often
managers becomes slaves to methods, and they follow them to the letter
because of the temptation to dodge their responsibility to think and be
accountable: they can blame the method. Or in the case of pure democratic
method, blame the team (You voted for it!). Methods can can easily encourage
the tolerance for design-by-committee type decisions.

So in the case of "how many alternatives", I'm a huge advocate of delegating
the design decisions to the point where a small group of people (possibly
one), can easily figure this out for themselves - based on the resources
they have, divided by the short ordered list of which design decisions are
most important. If no such list exists, they should be motivated to make
one.

If power is distributed well, you're less likely to need a "method".

-Scott

Scott Berkun
www.scottberkun.com

_____

From: Chauncey Wilson [mailto:chauncey.wilson at gmail.com]
Sent: Monday, January 19, 2009 11:44 AM
To: Scott Berkun
Cc: discuss at ixda.org
Subject: Re: [IxDA Discuss] How many alternatives, concepts, or sketches are
enough?

You make a good point though I didn't specifically mention equal voting at
all. You could have a small group who, as you say, have their necks on the
line or you could have private voting of the 10 top designers in the country
using polling software or you could generate criteria and have your small
group use the criteria as a starting point for a deeper discussion of the
type you suggest. You mention listing the criteria on the board which is a
great starting point, because many groups fail to explicitly identify
criteria that they are using (that method sounds like the QOC method -
Questions-Options-Criteria - that is described in the "design rationale"
literature.) \

19 Jan 2009 - 2:54pm
Chauncey Wilson
2007

Persuasion and passion are important and I'm seeing more references to
principles of persuasion in design discussions (for example the work of
Cialdini). In fact, a solid grounding in persuasion principles should be
part of our professional training. I wrote an essay on the use of persuasive
techniques for usability practitioners in 2007 issue of the
ACM interactions. Methods (or more accurately, the output of methods which
can be qualitative or quantitative) should, as you say, provide data and
interpretations for discussion and debate and different methods can even
provide different sides of the problem (method triangulation).

I profoundly dislike the autocratic application of "standard" methods and
like to consider how different methods can examine different angles to a
problem. For example, you wrote an excellent essay on how to run
brainstorming sessions. Group brainstorming is a complex social environment
and hard to do well. There is another technique called brainwriting (which
you might have written about) which can be used to gather input when groups
are shy or there are political concerns or you have a mix of old and new
people. The brainwriting method can complement group brainstorming and
often provides an outlet for those who may be anxious in a group setting.

So, methods and their output should be used to expose a range of issues,
contribute to debate and discussion, and support the triangulation of data
that will reduce risk to stakeholders and eventually customers.

I very much enjoyed your brainstorming write-up by the way and reference it
in a chapter that I've written on brainstorming, brainwriting, and
braindrawing.

thanks,
Chauncey

On Mon, Jan 19, 2009 at 3:01 PM, Scott Berkun <info at scottberkun.com> wrote:

> True - you didn't - Sorry for criticizing you for something you did not say
> :)
>
> My bias is against teams pretending to quantify the unquantifiable. I like
> opinions. I like things that designers believe but can not prove
> mathematically, but can explain through argument. Any decision making
> process that doesn't make use of conviction, persuasion and passion is one
> I
> doubt will work out well.
>
> Decision models/methods are great provided they're fodder - that they're
> used to help the discussion and debate, but not to replace it. Too often
> managers becomes slaves to methods, and they follow them to the letter
> because of the temptation to dodge their responsibility to think and be
> accountable: they can blame the method. Or in the case of pure democratic
> method, blame the team (You voted for it!). Methods can can easily
> encourage
> the tolerance for design-by-committee type decisions.
>
> So in the case of "how many alternatives", I'm a huge advocate of
> delegating
> the design decisions to the point where a small group of people (possibly
> one), can easily figure this out for themselves - based on the resources
> they have, divided by the short ordered list of which design decisions are
> most important. If no such list exists, they should be motivated to make
> one.
>
> If power is distributed well, you're less likely to need a "method".
>
> -Scott
>
> Scott Berkun
> www.scottberkun.com
>
> _____
>
> From: Chauncey Wilson [mailto:chauncey.wilson at gmail.com]
> Sent: Monday, January 19, 2009 11:44 AM
> To: Scott Berkun
> Cc: discuss at ixda.org
> Subject: Re: [IxDA Discuss] How many alternatives, concepts, or sketches
> are
> enough?
>
>
> You make a good point though I didn't specifically mention equal voting at
> all. You could have a small group who, as you say, have their necks on the
> line or you could have private voting of the 10 top designers in the
> country
> using polling software or you could generate criteria and have your small
> group use the criteria as a starting point for a deeper discussion of the
> type you suggest. You mention listing the criteria on the board which is a
> great starting point, because many groups fail to explicitly identify
> criteria that they are using (that method sounds like the QOC method -
> Questions-Options-Criteria - that is described in the "design rationale"
> literature.) \
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help
>

19 Jan 2009 - 5:33pm
Iain Barker
2009

As the person that created the slides Harry referred to, the key point
I was trying to make is that reaction to initial design sketches often
sets the design direction and constrains the design space within which
we explore potential solutions.

My intention with the slides was to warn practitioners of this, and
encourage that they actively try to keep the design space
unconstrained for as long as possible. It is difficult to do this,
especially if you get positive feedback from the first sketches you
share with colleagues/stakeholders/users.

As soon as we stop exploring different concepts and start iterating,
we are optimising an idea rather than looking for different
(innovative?) solutions.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=37356

19 Jan 2009 - 7:49pm
Scott Berkun
2008

Hi Ian - thanks for speaking up. I agree with the point you made in your
talk, regardless of how much I may have butchered it in this thread :)

Although I worry more about this: who are these sketches being shown to? And
why are the viewers of these sketches deciding the design direction?

I'm not asking for a fantasy land where designers have mind control powers
over clients and executives, but I am saying that when a sketch is shown,
the person showing the sketch should take responsibility for the
conversation it creates. What you describe is a common failing. But the only
person to recognize / prevent / solve the failing is the person who made the
sketch. They have more control over how that sketch is seen and interpreted
than anyone else.

The easy trick advertisers have known forever is never show one sketch to a
client. Always show three or five. Cynically they do this to make their own
favorite look good, but done more constructively it establishes, right away,
that there are multiple directions worthy of exploration.

-Scott

Scott Berkun
www.scottberkun.com

-----Original Message-----
From: discuss-bounces at lists.interactiondesigners.com
[mailto:discuss-bounces at lists.interactiondesigners.com] On Behalf Of iain
barker
Sent: Monday, January 19, 2009 3:34 PM
To: discuss at ixda.org
Subject: Re: [IxDA Discuss] How many alternatives, concepts,or sketches are
enough?

As the person that created the slides Harry referred to, the key point I was
trying to make is that reaction to initial design sketches often sets the
design direction and constrains the design space within which we explore
potential solutions.

19 Jan 2009 - 8:23pm
Michael Micheletti
2006

On Mon, Jan 19, 2009 at 3:13 AM, Jakub Linowski <jlinowski at gmail.com> wrote:

>
> <http://www.90percentofeverything.com/2009/01/14/why-you-shouldnt-rush-into-a-solution-too-quickly/>
> The question which I am wondering about then is how do we know how many
> alternatives are enough? How do we know we have enough sketches,
> alternatives or concepts before we begin choosing a satisfising solution.
>
>
My manager is asking this too, as I'm now working on my eleventh set of
concept sketches for a new software product. Probably several more ahead of
me still. Sigh.

But there's a telltale indicator that lets me know when I'm getting close.
It's when the design stays put for at least 24 hours with no one on the
design team, including me, wanting to change anything. I call it the 24 Hour
Rule. Maybe someone taught this to me - if so, thank you, whoever you were -
it's been helpful. Maybe it's original; I've been using it so long I can't
remember. The 24 Hour Rule means if you're still twitchy to change
something, you're not done with the design yet, even if engineers are
already building the product.

And I seem to recall a graphic design course where our instructor had us
bring in a hundred variations of whatever it is we were drawing. But maybe
we were bad and she wanted to punish us...

Michael Micheletti

Syndicate content Get the feed