Scenarios for multitouch displays?

1 Jun 2007 - 6:17am
7 years ago
16 replies
1209 reads
.pauric
2006

I'd like to address the critism that some on the list are not taking a
progressive view of Surface (inc all the prior art/technology). So
lets start finding problems for the tech to solve..

Assumption: we will have low cost, multitouch, small-large flat panel
displays to develop applications for by the end of the decade.

Question: what are these mass market applications, scenarios, problems?

For me, this type of human-machine interaction only works when the
machine is running an intelligent personal agent. Something that can
recognise faces in photos as it sucks images off your camera and tags
accordingly. Recognises the face or faces of the people standing in
front of it and presents content (news/videos) tailored to their
tastes. In short, when the machine can take up some slack of missing
a keyboard.

Comments

1 Jun 2007 - 8:15am
Trip O'Dell
2007

> Assumption: we will have low cost, multitouch, small-large flat panel
> displays to develop applications for by the end of the decade.
>
> Question: what are these mass market applications, scenarios,
> problems?
I think this is the wrong question. This is the same question that
all the B-school types were asking about the internet in the middle/
late '90s and is what led to so many ill-concieved, over-caffeinated,
internet startups. Its very difficult to tell what the mass market
appeal of a technology will be before its had time to have the tires
kicked.

Consider the ipod. In the eyes of many, a 5gb, $500 MP3 player with
one button was the very height of design folly.
Ipod was going to be the new Newton. What happened was a very
different story. The device became the catalyst for a new, very
disruptive class of technology that is changing the way people
perceive their "personal media" and computing in general.

Ipod helped gave rise to an entirely new class of mass market, DIY
media in the form of podcasts. Apple didn't invent, or even
anticipate podcasts when Ipod was in the 1st generation. In fairness
it really wasn't possible to anticipate podcasting as an emergent
behavior. Podcasting required several things to really take off:

1) An approachable, easy to use, device with a large capacity drive -
check, Ipod year: 2000

2) Cross platform support for the device with well designed media
management software (itunes) 2001

3) A uniform, trusted content distribution network - itunes music
store 2002 (arguably not possible without users being pre-conditioned
with Illegal peer to peer sharing)

4) A full range of devices and price points to bring a broad audience
to the marketplace (mini 2002 , shuffle 2003 , nano 2005)

5) Early adopter DIY community developing low cost, peer to peer
content (2002-ish)

6) Network support for free, user-submitted, syndicated media
(podcasting) Itunes 4.8 - 2005

My point is, the value of the ipod has grown organically and in
large part because of a network effect (the value of a network
compounds as it grows). User experience was critical to capture the
influentials and early adopters in the beginning. But ease of use
became less relevant as the early adopters and the ipod's price tag
(with its attendant status) made the device a cultural phenomenon.

I think multi-touch applications will grow in the same way. The first
thing everyone mentions when they see this kind of demo is "Minority
Report" because the movie is the only example of this kind
interaction in the public consciousness. There are going to be a lot
of Minority Report interface knock-offs before designers really
understand what the new affordances of this kind of an interface will
be.

I mostly design interactions for public and collaborative spaces
where we almost never assume a single user. I can already see many
different types of application for this kind of interaction.
Augmented board games, home automation and media control, boutique
retail, museums and public spaces, collaborative visualization, trade-
shows, transactional interfaces - I think the possibilities are only
limited by our imagination. Surface is a positive step in the
direction of making user interactions more social and collaborative.

Trip O'Dell
Interaction Designer
www.tripodell.com
------------------------------
"Specialization is for insects..."
-Robert A. Heinlein

1 Jun 2007 - 8:35am
Becubed
2004

> Question: what are these mass market applications, scenarios, problems?
>
> Assumption: we will have low cost, multitouch, small-large flat panel
> displays to develop applications for by the end of the decade.

Thanks for suggesting this, pauric; it's a good suggestion for shifting this
thread onto more positive ground.

I'd like to add one more assumption: that security issues have been resolved
to the extent that when you walk away from the table, it's a clean slate (so
to speak). No personal info crumbs left behind.

So a few ideas:

- Meeting room whiteboards. Here's one that I would personally love to see
today: imagine a large multi-touch wall. Interact with your hands, with pens
or pointing devices, with real-world objects; work on a blank surface, a
static image, or with an interactive application of some sort. A new breed
of productivity or creativity software could arise from this, designed to
help teams collaborate.

- Retail. Retail stores could go crazy with this, offering numerous ways for
people to explore and learn about inventory. Inset one into the front window
and static displays would seem quaint in comparison, as you could actually
*engage* passers-by.

- Planning transportation. A large interactive surface area could be a
pleasure to work with in comparison to current options. Imagine a
subway/metro station: plunk down your mobile phone or credit card; select
your destination(s), click to pay and get your ticket, pick up your
phone/card and walk away. Similar idea at airports and bus stations.

- Restaurants. Has anyone here NOT been frustrated at the difficulty of
splitting a table's bill among sub-groups of the diners? A large interactive
surface could make this a cakewalk (ahem). I believe this scenario was
illustrated in the Surface video, in fact. If cost is prohibitive, all
that's required is single device in the restaurant for anyone to use during
payment.

--
Robert Barlow-Busch
Terapath Inc.
bbb at terapath.net

1 Jun 2007 - 8:51am
.pauric
2006

I think its fair to counter your version of the iPod chain of events
with the points that there were personal music players before, as
there are computers preceding the Surface. But Apple's key to
success was to tell the story, people could relate to portable music
players in their pockets. Then in conjunction with itunes you had a
vertical solution. It isnt about the tech, its about the music.

Regarding the "B-school types (that) were asking about the internet
in the middle/ late'90s and is what led to so many ill-concieved,
over-caffeinated, internet startups." Hold up! they weren't asking
where's the story? Money was gushing, no one asked the right
questions and when dawn rose after the party it was a very bleak
landscape.

Now, we're all wise enough to be able to peer in to our personal
crystal balls and postulate on the vision, the story. Its a fools
errand to just sit here and say this is disruptive technology, a new
dawn is rising.

What is being disrupted? Kiosks?

Hasn't anyone got a realistic elevator pitch on this thing???

(fwiw: I'm not anti MS or pro Apple, I personally think the iPod is
a P.O.S. (but its got a killer story))

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://beta.ixda.org/discuss?post=16850

1 Jun 2007 - 9:21am
Taneem Talukdar
2005

Hi all,

Before I continue with the assumption that for this discussion, security
issues are moot, I just wanted to say that I'm not sure why security is such
a big deal to start with. It seems fairly certain that you would need some
kind of authentication system before you could do many of the things like
what the videos on the Surface website show. Kind of like how you log in to
computer accounts today. Maybe in some places it would use biometrics, in
others a username and password ... whatever. Are there any security issues
specific to a touch computer screen that wouldn't be addressed with logging
in / logging out etc?

As for product ideas -- I just spent a few minutes tapping my fingers on the
flat surface of my desk and moving around imaginary documents and emails ...
from this *extensive* testing, I think I'd be pretty comfortable with
removing my computer (and my phone) and replacing it with a Surface like
machine in my cubicle (what an unexciting application, I know).

Another good use would be at home where you would be able to use the
internet without "going on the computer". Just replace your regular living
room table with one of these (assuming that it's robust enough for this
environment). Then while relaxing in your living room, this table could
serve as everything from displaying the TV guide, displaying a virtual TV
remote (never lose the remote again!), allowing you to make phone calls
without needing a regular phone (VOIP!), to opening a webcam connection with
a friend, or a mini-browser window to check the weather.

I think that's the kind of thing that's quite realistic, and is the kind of
applications that Surface-like products are uniquely suited for. If this
thing runs Vista, and allows for USB connections, it would not be hard to
actually build a product around the table that does what I just described. I
think that's the bigger innovation here by Microsoft, rather than the actual
mechanics or concept of multi-touch.

On the flipside...

I also think it's tempting to ascribe applications to this that really,
could be easily done on regular computers (but people don't). For example,
the video on the MS site of the guy mapping a route on a map on Surface and
transferring it to a phone. You don't need a touch surface to do that. You
could just as easily click on two locations on a Google map and have it
transfer the route over to your phone (and maybe somewhere this application
already exists). But nobody seems to do that. I think this is analogous to
the kind of trap that some of the dotcoms fell into that Trip described.

Cheers,

Taneem Talukdar

On 6/1/07, pauric <radiorental at gmail.com> wrote:
>
> I think its fair to counter your version of the iPod chain of events
> with the points that there were personal music players before, as
> there are computers preceding the Surface. But Apple's key to
> success was to tell the story, people could relate to portable music
> players in their pockets. Then in conjunction with itunes you had a
> vertical solution. It isnt about the tech, its about the music.
>
> Regarding the "B-school types (that) were asking about the internet
> in the middle/ late'90s and is what led to so many ill-concieved,
> over-caffeinated, internet startups." Hold up! they weren't asking
> where's the story? Money was gushing, no one asked the right
> questions and when dawn rose after the party it was a very bleak
> landscape.
>
> Now, we're all wise enough to be able to peer in to our personal
> crystal balls and postulate on the vision, the story. Its a fools
> errand to just sit here and say this is disruptive technology, a new
> dawn is rising.
>
> What is being disrupted? Kiosks?
>
>
> Hasn't anyone got a realistic elevator pitch on this thing???
>
>
> (fwiw: I'm not anti MS or pro Apple, I personally think the iPod is
> a P.O.S. (but its got a killer story))
>
>
> . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Posted from the new ixda.org
> http://beta.ixda.org/discuss?post=16850
>
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> List Guidelines ............ http://listguide.ixda.org/
> List Help .................. http://listhelp.ixda.org/
> (Un)Subscription Options ... http://subscription-options.ixda.org/
> Announcements List ......... http://subscribe-announce.ixda.org/
> Questions .................. lists at ixda.org
> Home ....................... http://ixda.org/
> Resource Library ........... http://resources.ixda.org
>

1 Jun 2007 - 9:32am
Trip O'Dell
2007

> (fwiw: I'm not anti MS or pro Apple, I personally think the iPod is
> a P.O.S. (but its got a killer story))

LOL - I know, I've owned 6 of the damn things over the years. I just
can't help myself - SUCH a geek!

I can't agree with you more on the story aspect. I've actually
pitched these kinds of (Surface) interfaces in some of my work with
public spaces. Calling it all Kiosk is a little too dismissive in my
opinion. Kiosk (to me at least) conjures images of ATMs and "push to
play" video loops.

As a stand alone unit, Surface is a bit of a novelty. But when you
network units together so users can share simultaneous agency over a
virtual space in real time, Surface becomes a powerful collaborative
and social tool. Current online experience is heavily mediated and
abstracted through layers of interface mouse, keyboard, OS,
application UI and then finally content layout. Surface, (and
interfaces like it) cut through that clutter and provide an
opportunity to develop direct interaction with other users in more
powerfully intimate ways.

Since part of my background is game design, this kind of interface
has my head spinning with possibilities in terms of the games I can
design for this system. Games that wouldn't be possible, or if
possible not fun - in a more conventional physical interface. There
are entirely new audiences that will be introduced to video gaming as
a result:

1) networked card games with real cards
2) Bar games like air hockey that react to objects (like drinks) on
the surface by bouncing off them.
3) Sandbox environments with loose physics and rules where users can
make up their own games
4) augmented board games with new kinds of rules and strategies not
possible with a paper version.

Those are just a few ideas off the top of my head. The applications
for search, personal networking, communication, commerce, are equally
broad.

1 Jun 2007 - 9:59am
Trip O'Dell
2007

> On the flipside...
>
> I also think it's tempting to ascribe applications to this that
> really,
> could be easily done on regular computers (but people don't). For
> example,
> the video on the MS site of the guy mapping a route on a map on
> Surface and
> transferring it to a phone. You don't need a touch surface to do
> that. You
> could just as easily click on two locations on a Google map and
> have it
> transfer the route over to your phone (and maybe somewhere this
> application
> already exists).

I think context is important for this kind of application. Sure, its
easy for me to download directions into my phone and take it with
me...if I'm at home. If I'm on the road I need the following:

1) Either a phone with internet access (where I suffer through a
badly thought out UI to get the information I need) OR (more likely)
my laptop

2) A network connection that will let me access google maps

3) a means of easily transferring the information to my phone.

4) a willingness to go through all this hassle rather than ask
someone how to get where I want to do.

Where I see this kind of mapping application being useful, would be a
hotel lobby, or car rental agency. The security issue is a concern,
but that is more a limitation of phone design. It seems that
bluetooth on most devices is an all or nothing proposition. Why can't
there be levels of access permissions? EG - I choose how open I want
my phone to be in any given exchange.

Am I getting directions from a public terminal? The terminal has no
need to access my address book or personal data. Just the ability to
send directions to my map application.

Since Surface uses reflective camera occlusion, you can print
information on the back of the phone which is read by Surface when
you place it on the table - sort of like a limited WEP key to
establish contact. Surface sends an encryped signal to the phone,
phone wakes up, promps user to enter a password, a secure, but
limited connection is made. Directions are established and
transferred, user picks up phone. Session ends and erases personally
identifiable information from previous interaction and resets. GPS on
phone guides user off nearest cliff. Everyone is happy (cheers).

Based on the context of being in a strange city, and perhaps knowing
no more about my location than "you are here"- I would definitely use
this type of application if it were available. Its easier than
transferring information from my laptop (if I have it with me) and
more trustworthy than asking the clerk at the desk. I get a visual
representation of where I am at multiple scales with clues like local
landmarks in place.

1 Jun 2007 - 11:33am
Will Parker
2007

On Jun 1, 2007, at 7:21 AM, Taneem Talukdar wrote:

> Hi all,
>
> Before I continue with the assumption that for this discussion,
> security
> issues are moot, I just wanted to say that I'm not sure why
> security is such
> a big deal to start with. It seems fairly certain that you would
> need some
> kind of authentication system before you could do many of the
> things like
> what the videos on the Surface website show.
<SNIP>

First of all, thanks to Pauric for suggesting this fruitful topic.
For my part, I will stop referring specifically to the Microsoft
multi-touch table device unless the occasion warrants, and instead
use "multi-touch table" for this class of device.

I'd like to differentiate between three aspects of security here.

First, let's say that all aspects of physical and data security
inside the device are Solved. (BTW, I hereby dub this handy notion
Pauric's Assumption. It's got a beat, you can dance to it.)

The next aspect is protection of external data assets. A lot of
scenarios being discussed here involve laying devices containing
personal data (including wallets, key-ring data chits, and credit
cards) on a publicly-accessible table for extended periods of time.

So far, none of these scenarios has addressed how the multi-touch
table _reliably_ alerts the weary, jet-lagged passenger in the
Denver airport at 3:20 AM that he's paid for whatever and HE SHOULD
PICK UP HIS BLOODY ID CARD -- please, sir, NOW, before that shifty-
eyed gent who's been loitering nearby for the last four nights pays
another visit. Solving this issue is key to widespread adoption of
tab devices in electronic commerce.

Not so long ago on this list we thrashed out some ideas for ATM
security design. Much of what was said in that thread applies, but
not all. I'll come back to that in a separate thread when I have more
time, but one simple-minded requirement would be 'Customer must
remove the ID device from the table to finalize the payment.' Instead
of mouse-up, we can think of card-up events.

The third aspect lies in the public nature of multi-touch tables. As
myself and others have pointed out, the initial Microsoft positioning
for multi-touch tables is not in the privacy of the living room or
the conference room, but the wide-open spaces of an airport lounge.
Shoulder Surfers can now relax, because they can easily Read UR
EmailZ + ChartZ D00D from anywhere within 3 meters. (I predict a
small but thriving market in multi-touch table optical filters that
limit off-axis viewing.)

> As for product ideas -- I just spent a few minutes tapping my
> fingers on the
> flat surface of my desk and moving around imaginary documents and
> emails ...
> from this *extensive* testing, I think I'd be pretty comfortable with
> removing my computer (and my phone) and replacing it with a Surface
> like
> machine in my cubicle (what an unexciting application, I know).
>
> Another good use would be at home where you would be able to use the
> internet without "going on the computer". Just replace your regular
> living
> room table with one of these (assuming that it's robust enough for
> this
> environment). Then while relaxing in your living room, this table
> could
> serve as everything from displaying the TV guide, displaying a
> virtual TV
> remote (never lose the remote again!), allowing you to make phone
> calls
> without needing a regular phone (VOIP!), to opening a webcam
> connection with
> a friend, or a mini-browser window to check the weather.

Sooo, what you _really_ want is iPhone 2.0, but with a 30" screen and
weighing 50-65 pounds? 8-}

Personally, I've moved my personal computing and communication
environment entirely to portable devices. I don't want to be tied to
one location -- or work posture, for that matter. The only reason I
tolerate my TV is that I don't want to ask friends and family to
huddle around my laptop screen whenever we watch Dr. Who. Otherwise,
I want to be able to carry my digital environment wherever I decide
to go.

> I think that's the kind of thing that's quite realistic, and is the
> kind of
> applications that Surface-like products are uniquely suited for.

I agree, if we can get this down to:
- a sturdy, portable 17" form factor for general use,
- the current MS design for kiosk applications, and
- a drafting-table size for professional use.

Oh, and a whiteboard format to keep executives and Jack Bauer fans
happy.

> On the flipside...
>
> I also think it's tempting to ascribe applications to this that
> really,
> could be easily done on regular computers (but people don't).
<SNIP>

I'm not so enamored with the scenario of 'manually' (virtumanually?)
transferring data from device to device as shown in the multi-touch
table+camera+handheld scenario. Looks cool, but if I have a batch of
wireless devices (WiFi, Bluetooth, etc), why do the _mobile devices_
require a separate computer as a sort of 50-pound wireless dongle?
Why can't we make them smart enough to Just Work Together? Is it
really a stretch to make mobile devices that understand how to
silently share data appropriately with other devices owned by Self |
Family | Friends | Colleagues | Others?

Please note: I'm not saying any specific multi-touch table platform
is deficient in this area, but that this particular demo scenario --
for ANY multi-touch platform -- should be considered For
Entertainment Purposes Only.

- Will

Will Parker
wparker at ChannelingDesign.com

“I wish developing great products was as easy as writing a check. If
that were the case, then Microsoft would have great products.” -
Steve Jobs

1 Jun 2007 - 11:56am
Taneem Talukdar
2005

Great points. Just two thoughts:

I totally agree with the security issues but I think that some of these
apply in general to using computers in public, regardless of whether it's a
touch surface or not - one could also read your emails if you're using a
laptop at Denver airport, for example (the display on the surface screen is
not going to be any bigger or easier to read). You also need to remember to
take your card with you after paying at debit self-service stations (such as
in your grocery store or a gas stations). Smart designs there have been used
to address the issue, and smart designs like the one you have suggested can
be used for touch interfaces too. Thus, I'm pretty hopeful that security
issues are not going to be a showstopper with products like this.

Personally, I've moved my personal computing and communication
environment entirely to portable devices. I don't want to be tied to
one location -- or work posture, for that matter. The only reason I
tolerate my TV is that I don't want to ask friends and family to
huddle around my laptop screen whenever we watch Dr. Who. Otherwise,
I want to be able to carry my digital environment wherever I decide
to go.

Another thought I had is that, in the spirit of looking into the future, you
don't need to think of the Surface machine as storing your data. It's simply
an interface to the electronic warehouse of information and applications
available to you in your house. Thus, you don't need to be bogged down by
portability issues. Your information is accessible from multiple locations
(be it Surface in the living room, your laptop at work or your iPhone when
you're out jogging).

Cheers,
Taneem Talukdar

On 6/1/07, Will Parker <wparker at channelingdesign.com> wrote:
>
>
> On Jun 1, 2007, at 7:21 AM, Taneem Talukdar wrote:
>
> > Hi all,
> >
> > Before I continue with the assumption that for this discussion,
> > security
> > issues are moot, I just wanted to say that I'm not sure why
> > security is such
> > a big deal to start with. It seems fairly certain that you would
> > need some
> > kind of authentication system before you could do many of the
> > things like
> > what the videos on the Surface website show.
> <SNIP>
>
> First of all, thanks to Pauric for suggesting this fruitful topic.
> For my part, I will stop referring specifically to the Microsoft
> multi-touch table device unless the occasion warrants, and instead
> use "multi-touch table" for this class of device.
>
> I'd like to differentiate between three aspects of security here.
>
> First, let's say that all aspects of physical and data security
> inside the device are Solved. (BTW, I hereby dub this handy notion
> Pauric's Assumption. It's got a beat, you can dance to it.)
>
> The next aspect is protection of external data assets. A lot of
> scenarios being discussed here involve laying devices containing
> personal data (including wallets, key-ring data chits, and credit
> cards) on a publicly-accessible table for extended periods of time.
>
> So far, none of these scenarios has addressed how the multi-touch
> table _reliably_ alerts the weary, jet-lagged passenger in the
> Denver airport at 3:20 AM that he's paid for whatever and HE SHOULD
> PICK UP HIS BLOODY ID CARD -- please, sir, NOW, before that shifty-
> eyed gent who's been loitering nearby for the last four nights pays
> another visit. Solving this issue is key to widespread adoption of
> tab devices in electronic commerce.
>
> Not so long ago on this list we thrashed out some ideas for ATM
> security design. Much of what was said in that thread applies, but
> not all. I'll come back to that in a separate thread when I have more
> time, but one simple-minded requirement would be 'Customer must
> remove the ID device from the table to finalize the payment.' Instead
> of mouse-up, we can think of card-up events.
>
> The third aspect lies in the public nature of multi-touch tables. As
> myself and others have pointed out, the initial Microsoft positioning
> for multi-touch tables is not in the privacy of the living room or
> the conference room, but the wide-open spaces of an airport lounge.
> Shoulder Surfers can now relax, because they can easily Read UR
> EmailZ + ChartZ D00D from anywhere within 3 meters. (I predict a
> small but thriving market in multi-touch table optical filters that
> limit off-axis viewing.)
>
> > As for product ideas -- I just spent a few minutes tapping my
> > fingers on the
> > flat surface of my desk and moving around imaginary documents and
> > emails ...
> > from this *extensive* testing, I think I'd be pretty comfortable with
> > removing my computer (and my phone) and replacing it with a Surface
> > like
> > machine in my cubicle (what an unexciting application, I know).
> >
> > Another good use would be at home where you would be able to use the
> > internet without "going on the computer". Just replace your regular
> > living
> > room table with one of these (assuming that it's robust enough for
> > this
> > environment). Then while relaxing in your living room, this table
> > could
> > serve as everything from displaying the TV guide, displaying a
> > virtual TV
> > remote (never lose the remote again!), allowing you to make phone
> > calls
> > without needing a regular phone (VOIP!), to opening a webcam
> > connection with
> > a friend, or a mini-browser window to check the weather.
>
> Sooo, what you _really_ want is iPhone 2.0, but with a 30" screen and
> weighing 50-65 pounds? 8-}
>
> Personally, I've moved my personal computing and communication
> environment entirely to portable devices. I don't want to be tied to
> one location -- or work posture, for that matter. The only reason I
> tolerate my TV is that I don't want to ask friends and family to
> huddle around my laptop screen whenever we watch Dr. Who. Otherwise,
> I want to be able to carry my digital environment wherever I decide
> to go.
>
> > I think that's the kind of thing that's quite realistic, and is the
> > kind of
> > applications that Surface-like products are uniquely suited for.
>
> I agree, if we can get this down to:
> - a sturdy, portable 17" form factor for general use,
> - the current MS design for kiosk applications, and
> - a drafting-table size for professional use.
>
> Oh, and a whiteboard format to keep executives and Jack Bauer fans
> happy.
>
> > On the flipside...
> >
> > I also think it's tempting to ascribe applications to this that
> > really,
> > could be easily done on regular computers (but people don't).
> <SNIP>
>
> I'm not so enamored with the scenario of 'manually' (virtumanually?)
> transferring data from device to device as shown in the multi-touch
> table+camera+handheld scenario. Looks cool, but if I have a batch of
> wireless devices (WiFi, Bluetooth, etc), why do the _mobile devices_
> require a separate computer as a sort of 50-pound wireless dongle?
> Why can't we make them smart enough to Just Work Together? Is it
> really a stretch to make mobile devices that understand how to
> silently share data appropriately with other devices owned by Self |
> Family | Friends | Colleagues | Others?
>
> Please note: I'm not saying any specific multi-touch table platform
> is deficient in this area, but that this particular demo scenario --
> for ANY multi-touch platform -- should be considered For
> Entertainment Purposes Only.
>
> - Will
>
> Will Parker
> wparker at ChannelingDesign.com
>
> "I wish developing great products was as easy as writing a check. If
> that were the case, then Microsoft would have great products." -
> Steve Jobs
>
>
>

1 Jun 2007 - 11:57am
Becubed
2004

> Personally, I've moved my personal computing and communication
> environment entirely to portable devices. I don't want to be tied to
> one location -- or work posture, for that matter.

Building on your later point that devices should just Work Together... I'm
working on my laptop at the moment, which is on (gasp!) a table. It would be
*amazing* if that tabletop could become an extension of my computer's
virtual desktop.

I already have two screens on the go: the laptop's and an external 20"
widescreen. But the prospect of my whole desk surface being available has me
almost drooling. Now pair that with recognizing my cellphone (on the desk
beside me), my iPod (also on the desk), and we're rockin'.

And once a week a virtual hose would spray towards my potted plants sitting
on the table, reminding me to water the poor things.

--
Robert Barlow-Busch
Terapath Inc.
bbb at terapath.net

1 Jun 2007 - 7:44pm
Oleh Kovalchuke
2006

On 6/1/07, Will Parker <wparker at channelingdesign.com> wrote:

"...one simple-minded requirement would be 'Customer must
remove the ID device from the table to finalize the payment.' Instead
of mouse-up, we can think of card-up events."

This is an extension of "spring-loaded" security model.

I was thinking along the same lines: physical objects interacting with the
table should be equiped with spring-loaded button for sharing stuff from the
public "folder" in the object (phone, camera, whatever...). The device
should not be recognized by the table unless the user holds the button. This
approach could prevent some embarassing and imprudent scenarios. The button
should be of spring-n-lock-loaded variety actually - the lock is for use of
private table or for use with non-private devices.

--
Oleh Kovalchuke
Interaction Design is the Design of Time
http://www.tangospring.com/IxDtopicWhatIsInteractionDesign.htm

>
> On Jun 1, 2007, at 7:21 AM, Taneem Talukdar wrote:
>
> > Hi all,
> >
> > Before I continue with the assumption that for this discussion,
> > security
> > issues are moot, I just wanted to say that I'm not sure why
> > security is such
> > a big deal to start with. It seems fairly certain that you would
> > need some
> > kind of authentication system before you could do many of the
> > things like
> > what the videos on the Surface website show.
> <SNIP>
>
> First of all, thanks to Pauric for suggesting this fruitful topic.
> For my part, I will stop referring specifically to the Microsoft
> multi-touch table device unless the occasion warrants, and instead
> use "multi-touch table" for this class of device.
>
> I'd like to differentiate between three aspects of security here.
>
> First, let's say that all aspects of physical and data security
> inside the device are Solved. (BTW, I hereby dub this handy notion
> Pauric's Assumption. It's got a beat, you can dance to it.)
>
> The next aspect is protection of external data assets. A lot of
> scenarios being discussed here involve laying devices containing
> personal data (including wallets, key-ring data chits, and credit
> cards) on a publicly-accessible table for extended periods of time.
>
> So far, none of these scenarios has addressed how the multi-touch
> table _reliably_ alerts the weary, jet-lagged passenger in the
> Denver airport at 3:20 AM that he's paid for whatever and HE SHOULD
> PICK UP HIS BLOODY ID CARD -- please, sir, NOW, before that shifty-
> eyed gent who's been loitering nearby for the last four nights pays
> another visit. Solving this issue is key to widespread adoption of
> tab devices in electronic commerce.
>
> Not so long ago on this list we thrashed out some ideas for ATM
> security design. Much of what was said in that thread applies, but
> not all. I'll come back to that in a separate thread when I have more
> time, but one simple-minded requirement would be 'Customer must
> remove the ID device from the table to finalize the payment.' Instead
> of mouse-up, we can think of card-up events.
>
> The third aspect lies in the public nature of multi-touch tables. As
> myself and others have pointed out, the initial Microsoft positioning
> for multi-touch tables is not in the privacy of the living room or
> the conference room, but the wide-open spaces of an airport lounge.
> Shoulder Surfers can now relax, because they can easily Read UR
> EmailZ + ChartZ D00D from anywhere within 3 meters. (I predict a
> small but thriving market in multi-touch table optical filters that
> limit off-axis viewing.)
>
> > As for product ideas -- I just spent a few minutes tapping my
> > fingers on the
> > flat surface of my desk and moving around imaginary documents and
> > emails ...
> > from this *extensive* testing, I think I'd be pretty comfortable with
> > removing my computer (and my phone) and replacing it with a Surface
> > like
> > machine in my cubicle (what an unexciting application, I know).
> >
> > Another good use would be at home where you would be able to use the
> > internet without "going on the computer". Just replace your regular
> > living
> > room table with one of these (assuming that it's robust enough for
> > this
> > environment). Then while relaxing in your living room, this table
> > could
> > serve as everything from displaying the TV guide, displaying a
> > virtual TV
> > remote (never lose the remote again!), allowing you to make phone
> > calls
> > without needing a regular phone (VOIP!), to opening a webcam
> > connection with
> > a friend, or a mini-browser window to check the weather.
>
> Sooo, what you _really_ want is iPhone 2.0, but with a 30" screen and
> weighing 50-65 pounds? 8-}
>
> Personally, I've moved my personal computing and communication
> environment entirely to portable devices. I don't want to be tied to
> one location -- or work posture, for that matter. The only reason I
> tolerate my TV is that I don't want to ask friends and family to
> huddle around my laptop screen whenever we watch Dr. Who. Otherwise,
> I want to be able to carry my digital environment wherever I decide
> to go.
>
> > I think that's the kind of thing that's quite realistic, and is the
> > kind of
> > applications that Surface-like products are uniquely suited for.
>
> I agree, if we can get this down to:
> - a sturdy, portable 17" form factor for general use,
> - the current MS design for kiosk applications, and
> - a drafting-table size for professional use.
>
> Oh, and a whiteboard format to keep executives and Jack Bauer fans
> happy.
>
> > On the flipside...
> >
> > I also think it's tempting to ascribe applications to this that
> > really,
> > could be easily done on regular computers (but people don't).
> <SNIP>
>
> I'm not so enamored with the scenario of 'manually' (virtumanually?)
> transferring data from device to device as shown in the multi-touch
> table+camera+handheld scenario. Looks cool, but if I have a batch of
> wireless devices (WiFi, Bluetooth, etc), why do the _mobile devices_
> require a separate computer as a sort of 50-pound wireless dongle?
> Why can't we make them smart enough to Just Work Together? Is it
> really a stretch to make mobile devices that understand how to
> silently share data appropriately with other devices owned by Self |
> Family | Friends | Colleagues | Others?
>
> Please note: I'm not saying any specific multi-touch table platform
> is deficient in this area, but that this particular demo scenario --
> for ANY multi-touch platform -- should be considered For
> Entertainment Purposes Only.
>
> - Will
>
> Will Parker
> wparker at ChannelingDesign.com
>
> "I wish developing great products was as easy as writing a check. If
> that were the case, then Microsoft would have great products." -
> Steve Jobs
>
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> List Guidelines ............ http://listguide.ixda.org/
> List Help .................. http://listhelp.ixda.org/
> (Un)Subscription Options ... http://subscription-options.ixda.org/
> Announcements List ......... http://subscribe-announce.ixda.org/
> Questions .................. lists at ixda.org
> Home ....................... http://ixda.org/
> Resource Library ........... http://resources.ixda.org
>

2 Jun 2007 - 5:09pm
Will Parker
2007

On Jun 1, 2007, at 5:44 PM, Oleh Kovalchuke wrote:

> On 6/1/07, Will Parker <wparker at channelingdesign.com> wrote:
>
> "...one simple-minded requirement would be 'Customer must
> remove the ID device from the table to finalize the payment.' Instead
> of mouse-up, we can think of card-up events."
>
> This is an extension of "spring-loaded" security model.
>
> I was thinking along the same lines: physical objects interacting
> with the table should be equiped with spring-loaded button for
> sharing stuff from the public "folder" in the object (phone,
> camera, whatever...). The device should not be recognized by the
> table unless the user holds the button. This approach could prevent
> some embarassing and imprudent scenarios. The button should be of
> spring-n-lock-loaded variety actually - the lock is for use of
> private table or for use with non-private devices.

Excellent idea! Another variation would be to build a standard set of
lock/unlock gestures into the Surface multi-touch OS. Swirl a secure
device counterclockwise twice and place in the center of rotation to
unlock. To lock, pick up, swirl twice clockwise. Both accompanied by
appropriate standard indicators (TBD) for locked, unlocked, and lock-
change modes.

On the other hand, that would require either building motion sensors
in the device (adding expense), or trusting the multi-touch table to
truthfully indicate it had received an appropriate lock-change gesture.

But hey, at least one mobile device (whose name starts with 'i'
already has the appropriate motion sensors built-in. In that case,
require that the motion sensed by the two devices follows the same
profile in order to enable the change in lock status on the mobile
device.

With either Oleh's good solid idea or my flight of fantasy, we're
still assuming that the person who brings the mobile device to the
table is the legitimate owner. I think a publicly-visible keypad is a
Very Bad Idea for keying in passwords, but we do need _something_ to
authenticate access to secure devices and accounts. Signatures using
a stylus would work, but as Uncle Steve says, who wants a stylus when
you have fingers

Perhaps find a simple way for device owners to register their own
personal lock and unlock gestures? This could be securely stored in
the device itself or remotely in an networked ID repository.

What the multi-touch world needs now is a good standard for
converting a sequence of finger taps into a reliable biometric
signature.

- Will

Will Parker
wparker at ChannelingDesign.com

“I wish developing great products was as easy as writing a check. If
that were the case, then Microsoft would have great products.” -
Steve Jobs

5 Jun 2007 - 8:25pm
darlenepike
2007

Think about establishing a standard later, This could be really fun. People
could use the Chisenbop method to enter digits 0-9 with the fingers of one
hand, using a finger from the other hand or a palm-tap for zeros, Or someone
who plays a musical instrument could use the fingering of a series of chords
or a simple melody. It would be very individual and a lot easier to remember
than typical keyboard-entered username/passwords.

The screen could provide a simple crosshair for people to orient their
signatures -- three taps in the upper right quadrant being distinguishable
from three taps in the lower left. Because multiple data input points can be
captured with each press or combination of the fingerpresses, the interface
can be very simple while capturing passwords that are complex enough to be
secure.

or... umm... why wouldn't we write our signature with our fingers?

http://www.everything2.com/index.pl?node=Chisenbop

What the multi-touch world needs now is a good standard for
converting a sequence of finger taps into a reliable biometric
signature.

--
_____________________________________
Darlene Pike / Pike Design

Web coding for technically challenged visionaries™

web: www.PikeDesign.com
ph: 973-600-7113

5 Jun 2007 - 9:43pm
Trip O'Dell
2007

Great Thread -

"could use the Chisenbop method to enter digits 0-9 with the fingers of one
hand, using a finger from the other hand or a palm-tap for zeros"

The implication of a gesture interface is that the system adapts to the user, Chisenbop works for some people, but as a dyslexic, I found it really difficult to learn because I have difficulty determining my left and right side without thinking about it. I've adapted mnemonic tricks that help me determine the difference quickly, but never quickly enough to develop the muscle memory to make Chisenbop useful.

I think gesture systems will have to operate in the same way (adapting to the user and parsing context and memory to determine user intent). There will be a handful of basic commands common to every system, but the system will begin to learn combinations from individual users which are based on a particular context.

"Because multiple data input points can be
captured with each press or combination of the finger-presses, the interface
can be very simple while capturing passwords that are complex enough to be
secure.

>What the multi-touch world needs now is a good standard for
>converting a sequence of finger taps into a reliable biometric
>signature."

Actually - with systems like MS surface you don't even need that. The input uses infrared tracking and cameras to detect surface collisions, its the same technique already used for retinal scanning, and scanning a hand or finger print is really just a matter of camera resolution, exposure control and fidelity.

I think biometrics security on these sorts of interface will eventually be a passive constant which will also enable the system to determine individual users simultaneously interacting with the surface.

Exciting stuff.

Trip

7 Jun 2007 - 7:42am
darlenepike
2007

Yes -- "passive constant" -- ideally systems will recognize us as we are and
obviate our adapting in unnatural ways. I look forward to the day when
computer interfaces advance to the point where we can touch and speak
through them, and not need to type.

I think biometrics security on these sorts of interface will eventually be a
> passive constant which will also enable the system to determine individual
> users simultaneously interacting with the surface.
>

--
_____________________________________
Darlene Pike / Pike Design

Web coding for technically challenged visionaries™

web: www.PikeDesign.com
ph: 973-600-7113

7 Jun 2007 - 8:51am
.pauric
2006

Darlene: "ideally systems will recognize us as we are and obviate our
adapting in unnatural ways. I look forward to the day when computer
interfaces advance to the point where we can touch and speak through
them, and not need to type."

I'm of the opinion that machines arent human. Never will be and
therefore will never think exactly like us.

Thats why we have Enso in this day and age: http://www.humanized.com/

An interface is a meeting of two elements, usually in some middle
ground. Moving beyond the keyboard and its limitations does not mean
we'll one day be able to say "Yo! HAL, hold the email, hide the
pron, dim the lights and keep quiet, I got some company'

For me, interface design is part a story of mediating between the two
parties. Keeping that in mind helps me, at least, guide the user
gently in to a new 'language' instead of just going at the machine
with a nail gun.

I think most, or all, designers do this 'interfacing' but its worth
stating we're never going to make machines think like us. Humans
think like humans, machines will one day dream of electric sheep.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://beta.ixda.org/discuss?post=16850

5 Jul 2007 - 11:49am
Lynn Marentette
2007

Has anyone been working on ideas for something like the Surface in educational settings? I\'m a student, but I\'m also a school psychologist, so my focus is on developing interactive, user-friendly applications that would be useful in educational settings/situations.

We have Smartboards in K-12 settings, but they aren\'t quite \"right\", for a variety of reasons.

In the school setting, interacive displays need to be portable and multi-functional. They need to adjust to the size of the students and the purpose of the learning/communication activity.

For example, the user should be able to adjust the surface to the angle of a drafting table, place it vertically like an interactive whiteboard, or set it horizontally, like a table top.

The interactive surface\'s interoperability with hand-held devices and internet/network access would provide teachers with quite a bit of flexibility.

Syndicate content Get the feed