Estimating ROI for design

21 Feb 2005 - 8:41am
9 years ago
6 replies
699 reads
Petteri Hiisilä
2004

Hello everybody,

We're redesigning our corporate intranet using Cooper-ish methods. We're
doing it all: research, models, requirements, framework, design,
development support. 3000 employees were squeezed into one primary
persona and six others.

Based on our research, it's quite trivial to explain the qualitative
effects of the design for our corporate processes and people's workdays.

For example, the account managers can spend more time selling and less
time searching & sharing & editing different versions of the same
Powerpoint file. The new intranet will help the people & organization to
reach their goals. But...

How to turn this "soft" knowledge into hard currency?

Since our research was mostly qualitative, we don't have the
quantitative data, how much time the employees _really_ spend on
searching information from all the data repositories. We just know that
there was no bigger waste of time, so it has to be the biggest :) ...
The exact numbers weren't too useful for me as the designer, but they
sure are relevant to communicate the effects of the design.

The big bosses need to see ROI. Does anyone have books, articles or tips
on how to estimate the ROI for design?

Best,
Petteri

--
Petteri Hiisilä
Palveluarkkitehti / Interaction Designer /
Alma Media Interactive Oy / NWS /
+358505050123 / petteri.hiisila at almamedia.fi

"I was told there's a miracle for each day that I try"
- John Petrucci

Comments

21 Feb 2005 - 9:11am
Mike Baxter
2004

Petteri >>Does anyone have ... tips on how to estimate the ROI for design?

Done a few of these and from what you say < big bosses need to see ROI >
this is an exercise to be gone through rather than a genuine project
evaluation. In which case keep it simple and make a little data go a long
way! Here is how Gerry McGovern did some sums on measuring the value of
intranet content
http://www.gerrymcgovern.com/nt/2002/nt_2002_12_02_content_cost.htm

Another thought is to move away from ROI to Evidence of Value (EoV, as far
as your bosses are concerned!!). Instead of asking how much money will this
project make/save (which is usually more to do with conjecture than fact) as
you do in ROI, you begin by asking what value will derive from the project.
If this value was primarily focused on time savings, you could then
aggregate the total time saved by the organisation as a result of the
project. Evidence of Value would then compare the time required to complete
the project compared to the time saved as a result of the project. You could
even calculate a projected break-even period, so beloved of ROI-enthusiasts
(how long will it be before the time invested in this project is recovered
in time saved by the organisation).

Good luck!

Mike

21 Feb 2005 - 4:49pm
Marijke Rijsberman
2004

Hi Petteri,

I've looked and been unable to find anything apart from a number (users
spend 30% of their time searching for things), which had no other numbers to
back it up--so I didn't take note of where it was. I'm sure it varies
tremendously by type of job.

Failing to find ready-made quantitative data, I did some benchmarking on an
intranet project for a client that established some (putative) proxies for
productivity improvements from better information retrieval. We started out
with 45 representative tasks divided into two overlapping sets, one for
managers and one for non-manager employees, and then we measured completion
rates and times with 24 non-manager types and 13 managers for the "old"
intranet. Did the same thing all over again for the new intranet. I also
tried to measure:
- how long it took users to give up, figuring that if you are going to be
unsuccessful, you are much better off knowing that right away
- user satisfaction (on a scale from 1 to 10, with 9 and 10 counting as
promoters, 7 and 8 counting as neutral, 6 and below counting as
detractors--this is a culturally specific attempt to control for grade
inflation; the company I did this project for uses a similar measure for
their customer facing products)

I estimate that 23 percent of my braincells died in this most horribly
repetitive work that didn't prove anything besides. However, we did measure
a significant improvement in completion rates and times on all but 2 of the
tasks and a significant increase in user satisfaction. (The old intranet
didn't have a single promoter, lots of neutrals, and a few detractors; the
new intranet had a few promoters, lots of neutrals, and no detractors.)

I said that it didn't prove anything, because of course it didn't--it's
completely self-referential. There's nothing that tells you how it relates
to productivity. For all we know, the user's productivity went down because
they found all sorts of interesting things they were never able to find
before and got distracted from their work.

All the same, the big bosses were (somewhat) pacified. It's important to
note, however, that what really made an impression on them was the fact that
they saved more than $600K in the first year alone from decommissioning
servers that had been used to support a variety of sites that all got
integrated into the new design.

Regards,

marijke

<Petteri>
Since our research was mostly qualitative, we don't have the
quantitative data, how much time the employees _really_ spend on
searching information from all the data repositories. We just know that
there was no bigger waste of time, so it has to be the biggest :) ...
The exact numbers weren't too useful for me as the designer, but they
sure are relevant to communicate the effects of the design.</Petteri>

23 Feb 2005 - 11:48am
Lada Gorlenko
2004

PH> The big bosses need to see ROI. Does anyone have books, articles or tips
PH> on how to estimate the ROI for design?

Long overdue 2nd edition of "Cost-Justifying Usability: An update for
the Internet Age" by Bias and Mayhew will be published next month.
Reserve a copy - 1st edition has been pretty useful since 1994. 2nd
one is twice the size and I have lots of hope for it.

Lada

ps. Don't be put off by the "usability" term - it is NOT about testing
:-)

23 Feb 2005 - 11:52am
Elizabeth Buie
2004

Lada Gorlenko writes:

>ps. Don't be put off by the "usability" term - it is NOT about testing

I would revise that to say "Don't misunderstand the 'usability' term --
usability is NOT only about testing."

I'm glad to hear about the new edition of this book. Thanks for the
heads-up, Lada.

Elizabeth
--
Elizabeth Buie
Computer Sciences Corporation
Rockville, Maryland, USA
+1.301.921.3326

----------------------------------------------------------------------------------------
This is a PRIVATE message. If you are not the intended recipient, please
delete without copying and kindly advise us by e-mail of the mistake in
delivery. NOTE: Regardless of content, this e-mail shall not operate to
bind CSC to any order or other contract unless pursuant to explicit
written agreement or government initiative expressly permitting the use of
e-mail for such purpose.
----------------------------------------------------------------------------------------

23 Feb 2005 - 3:18pm
Lada Gorlenko
2004

>>ps. Don't be put off by the "usability" term - it is NOT about testing

EB> I would revise that to say "Don't misunderstand the 'usability' term --
EB> usability is NOT only about testing."

<whisper>
Sh-sh-sh... watch out for Big Brother... all goes according to plan...
designers don't like usability not being just about testing...
exercise caution when revising purposeful ambiguity...
</whisper>

Lada

23 Feb 2005 - 2:53pm
Austin Govella
2004

The Design Management Insitute offered a free download this month for
the article "Proving the practical power of design" where they measure
the effect design has on a company's market performance:
* http://tinyurl.com/52qt5

"....share prices of UK companies that use design effectively have
dramatically outperformed the rest of the market....by over 200%...."

They're referencing an article by the Design Council, "The impact of
design on stock market performance."
* http://tinyurl.com/63z7y

Both are available to download as PDFs.

Both the Design Council and Design Management Institute offer more
resources abut the business impacts of good design:

Design Management Institute:
* http://www.dmi.org

Design Council:
* http://www.designcouncil.org.uk

--
Austin Govella
Thinking & Making: IA, UX, and IxD
http://thinkingandmaking.com
austin at grafofini.com

Syndicate content Get the feed