Ecom Beta Site

3 Nov 2010 - 4:47pm
3 years ago
10 replies
667 reads
gthomas10
2010

Has anyone ever participated in a ecom site redesign where you created a beta environment that runs parallel to AND is accessible by users from the existing site?

The two environments would have the same functionalities and allow people to complete all the same tasks but Beta would have the new UI.  (Not an A/B situation. A completely functioning different version of the site.)

Users could try the beta environment but would always have the option to ‘eject’ from Beta and return to the ‘old’ UI.

The reason for considering this approach is that we want to - after internal testing - expriment with some fairly large differences in how the new UI works in comparison to the old.

To much for an e commerce site?
To risky from a user experience standpoint?
Dumb idea?

Any thoughts from having done this in the past?

Comments

3 Nov 2010 - 6:05pm
netwiz
2010

Yes. This is running at present on ba.com. If you have a booking with British Airways, you can log in to your booking to view details, make changes and requests etc - this is an area of the site called Manage My Booking.

  We redesigned the area, and did some standard usability testing to get the main usability issues identified, and then put it live. Initially, people logged in to the old version of MMB, and there was an invite on the page where they could switch to the new version - and switch back if they didn't like it. It was all the same functionality, but a different UI.

  When customers opted to use the new version, they got a popup which told them there was a feedback link on the new page. We received a large amount of invaluable feedback, which resulted in some fairly rapid changes.

  Over time we increased the number of people offered the new version, and now if you go to MMB in English, you will get the new version by default, still with the ability to feed back, and switch to the old version. Once we've made the new version entirely robust, and translated it, then we will present it in all languages, and eventually turn off the old version.

  It seems a sensible, even obvious approach to replacing critical functions on a site - although a big bang approach might be favoured if the current version is desperately in need of replacement.   Nick ba.com

On Wed, Nov 3, 2010 at 9:47 PM, gthomas10 <gthomas10@gmail.com> wrote:

Has anyone ever participated in a /ecom /site redesign where you created a beta environment that runs parallel to AND is accessible by users from the existing site?

The two environments would have the same functionalities and allow people to complete all the same tasks but Beta would have the new UI.  (Not an A/B situation. A completely functioning different version of the site.)

Users could try the beta environment but would always have the option to ‘eject’ from Beta and return to the ‘old’ UI.

The reason for considering this approach is that we want to - after internal testing - expriment with some fairly large differences in how the new UI works in comparison to the old.

To much for an e commerce site?
To risky from a user experience standpoint?
Dumb idea?

Any thoughts from having done this in the past?

(((Please leav
3 Nov 2010 - 6:05pm
Hugh Griffith
2007

Seems like a lot of unnecessary work. Wouldn't testing a low fidelity prototype with your customers accomplish the same thing?

Hugh Griffith
Visual / Interaction Designer
www.interactionhero.com

208-343-4420
Skype: hughgriffith
Gtalk: hugemm@gmail.com


On Wed, Nov 3, 2010 at 3:59 PM, gthomas10 <gthomas10@gmail.com> wrote:

Has anyone ever participated in a /ecom /site redesign where you created a beta environment that runs parallel to AND is accessible by users from the existing site?

The two environments would have the same functionalities and allow people to complete all the same tasks but Beta would have the new UI.  (Not an A/B situation. A completely functioning different version of the site.)

Users could try the beta environment but would always have the option to ‘eject’ from Beta and return to the ‘old’ UI.

The reason for considering this approach is that we want to - after internal testing - expriment with some fairly large differences in how the new UI works in comparison to the old.

To much for an e commerce site?
To risky from a user experience standpoint?
Dumb idea?

Any thoughts from having done this in the past?

(((Please
3 Nov 2010 - 6:53pm
Amy Silvers
2007

For confidentiality reasons, I can't say very much about my experience with this, but I have done it. It had to be done for technical reasons--there were a number of variables that had to be tested on the new architecture, and though we did a lot of that testing internally, to accurately test, we decided we had to involve real users.

It wasn't a great experience for some of our beta customers, though the vast majority had a seamless experience and noticed only the improvements in stability and speed. And while I can't really recommend it as an approach, it's doable, as long as you're aware of the risks.

My recommendations for anyone deciding to go that route:

  • test as much as possible through other methods before going to the beta, so that the beta isn't too much of an unknown quantity
  • communicate up front about the beta--set expectations, talk about what's changing and what isn't, etc.
  • give users the option of going back to the old version
  • as Nick says, increase the number of participants gradually (and start with a small number)
  • solicit feedback, and keep users posted about how you're incorporating that feedback
  • post a "known issues" list and keep it updated

 

--Amy

4 Nov 2010 - 8:28am
gthomas10
2010

Thanks for all the input.

Hugh - lo fi testing will be done and will help vet the interface ideas.  The entire UI paradigm of how users accomplish the main goal of the site is a drastic departure from what currently exists.

Our user base is so broad (spanning older people in their 80's all the way younger adults) that the general feeling is that the risk is to great to just completely switch to UI after testing.  (My client has a history of freaking out if basket size is affected to adversely.) Thus the more conservative approach of a Beta environment.

4 Nov 2010 - 12:05pm
Hugh Griffith
2007

How did the testing go? If people felt the new site was way better than the current one, then it's worth tickin' off a few grumps (there's always some).

If the results weren't really positive, then I might argue the redesign isn't necessary. Keep it the way it is.


Hugh Griffith
Visual / Interaction Designer
www.interactionhero.com
208-343-4420
Skype: hughgriffith
Gtalk: hugemm@gmail.com


On Thu, Nov 4, 2010 at 8:20 AM, gthomas10 <gthomas10@gmail.com> wrote:

Thanks for all the input.

Hugh - lo fi testing will be done and will help vet the interface ideas.  The entire UI paradigm of how users accomplish the main goal of the site is a drastic departure from what currently exists.

Our user base is so broad (spanning older people in their 80's all the way younger adults) that the general feeling is that the risk is to great to just completely switch to UI after testing.  (My client has a history of freaking out if basket size is affected to adversely.) Thus the more conservative approach of a Beta environment.

((
4 Nov 2010 - 1:05pm
gthomas10
2010

Hi Hugh

testing is not complete. I am pulling together some supporting information on the beta topic as we start to figure out the direction we want to go.

I see your point about tickin off grumps. My client can be prone to freak outs if basket sizes shrink hence the rational behind this idea.

Greg

On Thu, Nov 4, 2010 at 12:21 PM, Hugh Griffith <hgriffith@vfs.com> wrote:

How did the testing go? If people felt the new site was /way/ better than the current one, then it's worth tickin' off a few grumps (there's always some).

If the results weren't really positive, then I might argue the redesign isn't necessary. Keep it the way it is.

Hugh Griffith
Visual / Interaction Designer
www.interactionhero.com [1]
208-343-4420
Skype: hughgriffith
Gtalk: hugemm@gmail.com [2]

On Thu, Nov 4, 2010 at 8:20 AM, gthomas10 <gthomas10@gmail.com [3]> wrote:

Thanks for all the input.

Hugh - lo fi testing will be done and will help vet the interface ideas.  The entire UI paradigm of how users accomplish the main goal of the site is a drastic departure from what currently exists.

Our user base is so broad (spanning older people in their 80's all the way younger adults) that the general feeling is that the risk is to great to just completely switch to UI after testing.  (My client has a history of freaking out if basket size is affected to adversely.) Thus the more conservative approach of a Beta environment.

((

((
4 Nov 2010 - 5:05pm
netwiz
2010

Greg, I think your client is entirely justified in freaking out if basket size is impacted, if it means that he makes less money. We're not paid to make less monly for our companies or clients.   Hugh, you will learn a lot lot more from a live beta than from testing low fi prototypes. Lot os real people in real situations wanting to do real things will give you better information and feedback than low fi. But you need to do the low fi first to get it as close as you can - which we did.

On Thu, Nov 4, 2010 at 6:54 PM, gthomas10 <gthomas10@gmail.com> wrote:

Hi Hugh

testing is not complete. I am pulling together some supporting information on the beta topic as we start to figure out the direction we want to go.

I see your point about tickin off grumps. My client can be prone to freak outs if basket sizes shrink hence the rational behind this idea.

Greg

On Thu, Nov 4, 2010 at 12:21 PM, Hugh Griffith <hgriffith@vfs.com [1]> wrote:

How did the testing go? If people felt the new site was /way/ better than the current one, then it's worth tickin' off a few grumps (there's always some).

If the results weren't really positive, then I might argue the redesign isn't necessary. Keep it the way it is.

Hugh Griffith
Visual / Interaction Designer
www.interactionhero.com [2] [1]
208-343-4420
Skype: hughgriffith
Gtalk: hugemm@gmail.com [3] [2]

On Thu, Nov 4, 2010 at 8:20 AM, gthomas10 <gthomas10@gmail.com [4] [3]> wrote:

Thanks for all the input.

Hugh - lo fi testing will be done and will help vet the interface ideas.  The entire UI paradigm of how users accomplish the main goal of the site is a drastic departure from what currently exists.

Our user base is so broad (spanning older people in their 80's all the way younger adults) that the general feeling is that the risk is to great to just completely switch to UI after testing.  (My client has a history of freaking out if basket size is affected to adversely.) Thus the more conservative approach of a Beta environment.

((

((

(((
4 Nov 2010 - 8:05pm
gthomas10
2010


Hope that clarifies my rational.

GT

On Thu, Nov 4, 2010 at 5:06 PM, netwiz <netwiznick@gmail.com> wrote:

Greg, I think your client is entirely justified in freaking out if basket size is impacted, if it means that he makes less money. We're not paid to make less monly for our companies or clients.
 
Hugh, you will learn a lot lot more from a live beta than from testing low fi prototypes. Lot os real people in real situations wanting to do real things will give you better information and feedback than low fi. But you need to do the low fi first to get it as close as you can - which we did.

On Thu, Nov 4, 2010 at 6:54 PM, gthomas10 <gthomas10@gmail.com [1]> wrote:

Hi Hugh

testing is not complete. I am pulling together some supporting information on the beta topic as we start to figure out the direction we want to go.

I see your point about tickin off grumps. My client can be prone to freak outs if basket sizes shrink hence the rational behind this idea.

Greg

On Thu, Nov 4, 2010 at 12:21 PM, Hugh Griffith <hgriffith@vfs.com [2] [1]> wrote:

How did the testing go? If people felt the new site was /way/ better than the current one, then it's worth tickin' off a few grumps (there's always some).

If the results weren't really positive, then I might argue the redesign isn't necessary. Keep it the way it is.

Hugh Griffith
Visual / Interaction Designer
www.interactionhero.com [3] [2] [1]
208-343-4420
Skype: hughgriffith
Gtalk: hugemm@gmail.com [4] [3] [2]

On Thu, Nov 4, 2010 at 8:20 AM, gthomas10 <gthomas10@gmail.com [5] [4] [3]> wrote:

Thanks for all the input.

Hugh - lo fi testing will be done and will help vet the interface ideas.  The entire UI paradigm of how users accomplish the main goal of the site is a drastic departure from what currently exists.

Our user base is so broad (spanning older people in their 80's all the way younger adults) that the general feeling is that the risk is to great to just completely switch to UI after testing.  (My client has a history of freaking out if basket size is affected to adversely.) Thus the more conservative approach of a Beta environment.

((

((

(((

(((
21 Nov 2010 - 11:47pm
nhoh
2004

Sorry to be so late with a comment, but the one example that comes to mind is Zappos' Zeta site. They had that up and running with a different nav, search results and product pages, I don't know if they had alternate versions of the checkout flow though.

I guess I'm curious why you want to do a beta versus doing an A/B test? Are you going to promote the beta site for folks to test or will you funnel some portion of users to the Beta site and inform them that they can pull the cord if they want to?

Hope this helps!

Cheers,

Nick Hoh

22 Nov 2010 - 10:05am
gthomas10
2010

Thanks Nick

To answer your question; I am considering the Beta route over A/B because what will exist in the beta site will be drastically different than what exists right now.

GT

On Sun, Nov 21, 2010 at 11:29 PM, nhoh <nickhoh@yahoo.com> wrote:

Sorry to be so late with a comment, but the one example that comes to mind is Zappos' Zeta site. They had that up and running with a different nav, search results and product pages, I don't know if they had alternate versions of the checkout flow though.

I guess I'm curious why you want to do a beta versus doing an A/B test? Are you going to promote the beta site for folks to test or will you funnel some portion of users to the Beta site and inform them that they can pull the cord if they want to?

Hope this helps!

Cheers,

Nick Hoh

Syndicate content Get the feed