on research.

13 Jul 2006 - 1:14pm
8 years ago
8 replies
385 reads
mariaromera
2005

Just want to point out that the thing about research is that you can criticize it, replicate it, and/or improve upon it. This relies, of course, on a basic assumption that everyone involved (researchers and critics) understands what research is. Are there a lot of people out there masquerading as User Researchers who have never taken Research Methods or Statistics? Maybe... I don't know the answer to that, but I sure hope not.

I'm talking about giving basic background, stating a hypothesis, describing your methods of testing it, your results, and your conclusions. Now, maybe user research isn't always following the old formula. I know that some methods follow a more anthropological field study approach, rather than cognitive psychology's hypothesis testing. But both use research methods. There are times when you can break the rules, but you should *know* that you are breaking them.

I do think that if more people had an understanding of quantitative and qualitative statistics and what inferences you can and cannot draw from them, there would be FAR reaching consequences not just for research and design, not just for science, but for society and politics as well!

Yes, yes, all of you fabulous designers out there have good intuition and come up with great ideas sometimes without research. Sometimes we don't have the luxury of research. But think how much better we would all be if we can use it when we have the chance. And there is no better place for a junior designer to start than looking at data.

cheers,
maria

---------------------------------
Sneak preview the all-new Yahoo.com. It's not radically different. Just radically better.

Comments

13 Jul 2006 - 1:53pm
Mark Schraad
2006

On Jul 13, 2006, at 1:14 PM, maria romera wrote:

> [Please voluntarily trim replies to include only relevant quoted
> material.]
>
> Just want to point out that the thing about research is that you
> can criticize it, replicate it, and/or improve upon it. This
> relies, of course, on a basic assumption that everyone involved
> (researchers and critics) understands what research is. Are there
> a lot of people out there masquerading as User Researchers who
> have never taken Research Methods or Statistics? Maybe... I don't
> know the answer to that, but I sure hope not.

The implication inherent in your qualifying statement "Research
Methods or Statistics" leads me to believe that you are a positivist.
There are many forms of research that are not quantifiable, yet are
valid and evidentiary.

> I'm talking about giving basic background, stating a hypothesis,
> describing your methods of testing it, your results, and your
> conclusions. Now, maybe user research isn't always following the
> old formula. I know that some methods follow a more
> anthropological field study approach, rather than cognitive
> psychology's hypothesis testing. But both use research methods.
> There are times when you can break the rules, but you should
> *know* that you are breaking them.\

Absolutely!

> I do think that if more people had an understanding of
> quantitative and qualitative statistics and what inferences you
> can and cannot draw from them, there would be FAR reaching
> consequences not just for research and design, not just for
> science, but for society and politics as well!

There is a common misunderstanding that ethnography and cultural
anthropology are strictly qualitative and that is incorrect. Many
ethnographers have used quantitative methodologies and drawn
conclusion from that data.

One of the problems in social sciences is that when they play by
science methods fro research they fail. The social sciences need to
grow into a new process and testing vanacular - particularly
sociology and behavioral psychology. As an example, try finding a
control group to do an analysis of peculiar behavior. Standard
scientific methods break down in self evaluation and abstractions
such as motivation and the analysis of the non-conscious though
patterns. Though new brain scanning technologies are often revealing,
most of this falls on the cognitive side.

> Yes, yes, all of you fabulous designers out there have good
> intuition and come up with great ideas sometimes without
> research. Sometimes we don't have the luxury of research. But
> think how much better we would all be if we can use it when we
> have the chance. And there is no better place for a junior
> designer to start than looking at data.

Often the lack of research is ignorance, sometimes as designers we
are selfish with "our" vision, sometimes we convince ourselves that
we already know what the user needs or want. But the most common
problem is the lack of resources, either time or money, and that
support must come from the top. Even doing semi "proper" research and
presenting the evidence, rational and results can provide the sort of
results that infect those at the top of an organization. These are
the people designers need to be better at communicating with. They
will, once presented with return on investment (ROI) will fund the
research and allow the time to do things right.

>
> cheers,
> maria
>
>
>
>
> ---------------------------------
> Sneak preview the all-new Yahoo.com. It's not radically different.
> Just radically better.
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> List Guidelines ............ http://listguide.ixda.org/
> List Help .................. http://listhelp.ixda.org/
> (Un)Subscription Options ... http://subscription-options.ixda.org/
> Announcements List ......... http://subscribe-announce.ixda.org/
> Questions .................. lists at ixda.org
> Home ....................... http://ixda.org/
> Resource Library ........... http://resources.ixda.org

13 Jul 2006 - 2:32pm
John Schrag
2005

On Jul 13, 2006, at 1:14 PM, maria romera wrote:
> Often the lack of research is ignorance, sometimes
> as designers we are selfish with "our" vision, sometimes
> we convince ourselves that we already know what the user
> needs or wants.

One of the interesting things about our in-house User Experience team is
that all of us are generalists. We all do research, design, and
validation. It's really difficult to hire people for our team, though.
We find that many people with a strong design 'vision' (the 'inner
voice' as one of my colleagues describes it) are really terrible at
observation, whether in a usability test or during contextual inquiry.
They come back having seen only what they expected to see, so
observational research really is wasted on them.

Similarly, we find that many people with really insightful observational
skills lack that driving design vision. They can see the problems, but
don't have the insight on how to solve them creatively. We look for
people who have a strong inner voice, but also the ability to tell that
voice to "shut the hell up while I'm listening".

-john

User Experience Team,
Autodesk M&E division
Toronto

13 Jul 2006 - 2:59pm
Mark Schraad
2006

Sounds like you are looking for "T" shaped people who can bridle
their ego. Exactly right - and rare. I have found that young
designers right out of school have much more success with this dual
role than older print designers (nothing personal here - that is my
background from years ago).

M

On Jul 13, 2006, at 2:32 PM, John Schrag wrote:

> [Please voluntarily trim replies to include only relevant quoted
> material.]
>
>
> On Jul 13, 2006, at 1:14 PM, maria romera wrote:
>> Often the lack of research is ignorance, sometimes
>> as designers we are selfish with "our" vision, sometimes
>> we convince ourselves that we already know what the user
>> needs or wants.
>
> One of the interesting things about our in-house User Experience
> team is
> that all of us are generalists. We all do research, design, and
> validation. It's really difficult to hire people for our team,
> though.
> We find that many people with a strong design 'vision' (the 'inner
> voice' as one of my colleagues describes it) are really terrible at
> observation, whether in a usability test or during contextual inquiry.
> They come back having seen only what they expected to see, so
> observational research really is wasted on them.
>
> Similarly, we find that many people with really insightful
> observational
> skills lack that driving design vision. They can see the problems,
> but
> don't have the insight on how to solve them creatively. We look for
> people who have a strong inner voice, but also the ability to tell
> that
> voice to "shut the hell up while I'm listening".
>
> -john
>
> User Experience Team,
> Autodesk M&E division
> Toronto
>
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> List Guidelines ............ http://listguide.ixda.org/
> List Help .................. http://listhelp.ixda.org/
> (Un)Subscription Options ... http://subscription-options.ixda.org/
> Announcements List ......... http://subscribe-announce.ixda.org/
> Questions .................. lists at ixda.org
> Home ....................... http://ixda.org/
> Resource Library ........... http://resources.ixda.org

13 Jul 2006 - 3:57pm
Robert Hoekman, Jr.
2005

> Are there a lot of people out there masquerading as User Researchers who
> have never taken Research Methods or Statistics? Maybe... I don't know the
> answer to that, but I sure hope not.

So you can only do effective research if you took a college course on it?
And if you didn't, you shouldn't bother researching users?

-r-

13 Jul 2006 - 5:55pm
mariaromera
2005

>There are many forms of research that are not quantifiable, yet are
>valid and evidentiary.

absolutely!

>There is a common misunderstanding that ethnography and cultural
>anthropology are strictly qualitative and that is incorrect. Many
>ethnographers have used quantitative methodologies and drawn
>conclusion from that data.

I agree.

>One of the problems in social sciences is that when they play by
>science methods fro research they fail. The social sciences need to
>grow into a new process and testing vanacular - particularly
>sociology and behavioral psychology. As an example, try finding a
>control group to do an analysis of peculiar behavior.

There are different methods, though, for doing research without a control group. And sometimes you can come up with a new way to measure things... doesn't always have to be a formula, there's still room for creativity as long as it's something others can replicate.

>Even doing semi "proper" research and
>presenting the evidence, rational and results can provide the sort of
>results that infect those at the top of an organization. These are
>the people designers need to be better at communicating with. They
>will, once presented with return on investment (ROI) will fund the
>research and allow the time to do things right.

Yes, or at least you can get more leeway to research your most urgent questions. Although it can be tough to have to fight for these resources, I suppose it does keep you focused on the design questions... otherwise we would be academics :)

m

---------------------------------
How low will we go? Check out Yahoo! Messenger’s low PC-to-Phone call rates.

13 Jul 2006 - 5:59pm
mariaromera
2005

"Robert Hoekman, Jr." <rhoekmanjr at gmail.com> wrote:
>>Are there a lot of people out there masquerading as User >>Researchers who have never taken Research Methods or >>Statistics? Maybe... I don't know the answer to that, but I sure >>hope not.
>So you can only do effective research if you took a college course on >it? And if you didn't, you shouldn't bother researching users?

Robert, was my comment really so black and white?
If I asked you if you would hire someone to code C++ without a college degree in Computer Science, wouldn't I be oversimplifying? But it would sure make you feel good to know they had the right background.

---------------------------------
Do you Yahoo!?
Next-gen email? Have it all with the all-new Yahoo! Mail Beta.

13 Jul 2006 - 6:03pm
Steve Portigal
2004

maria romera wrote:
>
>"Robert Hoekman, Jr." <rhoekmanjr at gmail.com> wrote:
>>>Are there a lot of people out there masquerading as User
>>>Researchers who have never taken Research Methods or >>Statistics?
> Maybe... I don't know the answer to that, but I sure >>hope not.
>>So you can only do effective research if you took a college course
>on >it? And if you didn't, you shouldn't bother researching users?
>
>
> Robert, was my comment really so black and white?
> If I asked you if you would hire someone to code C++ without a
>college degree in Computer Science, wouldn't I be oversimplifying?
>But it would sure make you feel good to know they had the right
>background.

But all professions are not equal. Some have very clear
certifications, and educational backgrounds, and some don't.

I wouldn't ask someone to remove my appendix unless I knew they had
accredited medical training. I might ask someone to change my oil
without finding out if they've completed a Community College course
on auto engine repair.

I mean, I can come up with silly examples in either direction all
day; the point being that the comparisons are shaky, at best.

Steve Portigal -- http://www.portigal.com
blog is now at http://www.portigal.com/blog

13 Jul 2006 - 6:16pm
Katie Albers
2005

At 3:59 PM -0700 7/13/06, maria romera wrote:
>[Please voluntarily trim replies to include only relevant quoted material.]
>
>
>
>"Robert Hoekman, Jr." <rhoekmanjr at gmail.com> wrote:
>>>Are there a lot of people out there masquerading as User
>>>>>Researchers who have never taken Research Methods or
>>>>>Statistics? Maybe... I don't know the answer to that, but I
>>>sure >>hope not.
>>So you can only do effective research if you took a college course
>>on >it? And if you didn't, you shouldn't bother researching users?
>
>
> Robert, was my comment really so black and white?
> If I asked you if you would hire someone to code C++ without a
>college degree in Computer Science, wouldn't I be oversimplifying?
>But it would sure make you feel good to know they had the right
>background.

Well...you did say that people who consider themselves user
researchers should have at least a course in Research Methods and one
in Statistics...so...yes...it necessarily follows that people who
have neither (or only one or the other) cannot properly be considered
User Researchers. And that's a very black and white statement.

I think your comparison about the college degree in computing is
flawed at best. A course isn't comparable to a major. But it, for the
purposes of this conversation, we assume that they are
comparable...I've known brilliant C++ coders who taught themselves
and took neither a course in programming nor a major in Computer
Science. Would I feel better hiring the guy with the academic
credentials? Nope.

You may be trying to get a debate going here, in which case I'd have
to say that you're making some very broad and unfounded assumptions
and probably insulting lots of people here as much as you're
insulting me.

kt

Syndicate content Get the feed