, , ,

SXSW Interactive – 2012


This was my first experience at SXSW, so I had only a vague idea of what I was getting myself into.  My first glance at the session options beforehand made it seem like it was going to be overwhelming, and, well, it was.  For any given hour of the day there were probably 30+ things you could choose from to do.  How do you choose?  Serendipity, baby.

This conference was different from the usual technical and more academic conferences I’ve been to in the past.  There was a lot of hand-wavey, touchy-feely stuff going on, which took a little bit of adjustment to accept, but that was sort of the point of the conference.  It does, in a lot of ways, represent the landscape of our times in its overabundance of information being consumed by the masses in this chaotic, yet somehow self-ordering way.  My hardened left brain thinking I usually apply to this sort of context just wasn’t particularly appropriate.

But to be analytical anyway…  the conference seemed to be about:

  • Getting inspired, being creative, energizing yourself to make the next big thing
  • Meeting other people who you can help and who will help you on this path

There were numerous references to:

Anyway, below are the relatively raw notes I took for the sessions I attended, and maybe a few other things I saw.

Thanks to John Catalano at ServiceMesh for making it possible for me to attend this year!

Designing For Context

Microsoft did a number of studies in 1995 that led to “clippie”, using bayesian logic to detect when a user is getting frustrated with the UI. Most unfortunate.

Advice: Reject 1:1 mapping between contexts (e.g. between desktop version and mobile version of app)

Things to pay attention to when designing:
– time
– ecosystem
– location
– form & tech
– brand & relationships


Has to do with time available and time applied to interaction and engagement.

New York Times performed some research where they would monitor the site as people used it, and would go so far as to call up users while they were using the site to get their feedback on what they were trying to do, what was frustrating them, etc. Would ask questions about what kind of tasks they were wanting to be able to do, and what kind of time they had or needed to perform these tasks.


E.g. a classroom environment has certain needs you wouldn’t see in other environments.

The Mint (financial software) site did studies, collecting usage metrics on desktop vs mobile. Some users would see only one of the interfaces, so had to determine the key features that needed to be there on both. Found also that development of the mobile app ended up influencing change in the desktop app.

Discrepancy between Android vs iOS… android is made for scalable graphics (so many device possibilities) vs iOS where the target resolution is known. you get better graphics with the known environment, though some recent work with android is getting better.


victoria’s secret apps made by razorfish… made iphone & ipad apps completely different from each other because of how they get used. ipad usually used at home while planning, iphone usually used on site where things like reading a upc code come in handy. considered some form of augmented reality.

tax online for accountants by intuit… in office studies found that people would be posed questions on the go and would have to go back to their desks to get information. just having the app on an ipad helped loads.

ability to live stream over mobile in the moment in general is compelling…

you have to consider testing outdoors… found that users couldn’t read the screen outside.

Form & Technology

consider the screen and the input method.

e.g. tax application can take a picture of your W2, which sounds good, and people like it, but it actually takes longer than just keying the information in.

when porting to mobile from desktop or whatever…
• make a list of the key features
• make a list of device capabilities
==> then think of all the cool shit where they intersect


Formation and effect on user happiness.
e.g. bedsider.com developed a sex-positive brand for women. including segments like “douche & don’ts – misconceptions”


There’s been a general transition to touch screens. Some good, some bad, some steps forward, some steps back. Even Apple’s back button on the iPad was iffy.

Consider Fitt’s Law, that the further away an object is, the harder it is to hit. In UI… let people be lazy.

e.g. a pinball game on the ipad has no buttons, and it doesn’t need them. a fair amount of the real estate can be used for gestures without requiring the user to have good “aim”.

however, there’s room for development of a standard “gesture vocabulary”.

e.g. iOS5 introduces a 5-finger swipe that can be used to slide the back-button bar out, but who’s going to figure that out intuitively? what if you don’t have 5 fingers?

gestures are today’s keyboard shortcuts.
can we use the entire screen in general? big screens = big gestures.

“buttons are a hack”
even a light switch is a hack for turning on a light, though it’s a lot better than grabbing a chair and screwing the bulb in whenever you needed light

windows 8 is touting that it’s all touch based. how good it is at that remains to be seen.

twitter for the ipad has no back button. it has a swipable, semi-viewable stack that’s pretty intuitive by being spatially metaphorical.

however, there’s weak support for gestures on the web; e.g. limitations of javascript support.
jquery mobile and sencha touch add more support.
as does touchy.js

some good samples to look at:
– bit.ly/ikeatouch (ui without stereotypical controls)
– bit.ly/ios-clear (no buttons except for keyboard)

in general… you end up “playing the UI” instead of “using” it.
employing a bit of natural physical understanding.

example shown of a paint program that, instead of giving the option to change the pen size, gives you the option to change the image size (thus your finger input remains the same, but is smaller or bigger depending on the overall image zoom).

employ Clarity over Density.
do the opposite of a swiss army knife.
resist putting everything in and making your UI too complicated.
or so much that it’s no longer usable.

how do you help people know what they CAN do?
– resemble the physical action
– resemble what’s known on the desktop
– introduce visual hints

help them make the transition from learning to… muscle memory.

(injected funny image of an OCD cutting board)

suggesting reading: “Living with Complexity”
some social conventions are uncertain, even to people who know their own… others might not.
e.g. ambiguous salt & pepper shakers (one with one hole, another with 10… which is the salt?). you can label them explicitly, or you can make the contents transparent.
same with your UI.
can use thumbnails to make it transparent to the point that the medium becomes the message.

how would you teach gestures.
how-to instruction book? NOOOO.
– user hasn’t seen the app yet
– makes it seem hard
– no one reads it
(showed a funny spoof of a medieval transition from scrolls to books)
– nobody reads the manual
– people are impatient
– instructions get in the way
(showed bit.ly/our-choice, which highlighted a really good interactive app pushed by al gore)

nature doesn’t have instructions.
so… rely on that physicality.

the new iOS5’s calendar took a step backward. great visual metaphor for a physical calendar. you’d expect to be able to swipe to get to the next day, but nooo… you have to tap a small button on the button.

embrace the metaphor.
but also embrace the tech.

“Sydney Morning Herald” has done a good job of embracing the tech along with the metaphor by allowing the user to see ahead to the titles on other pages.

iphone voice memo app? there only needs to be one big button to record. that’s it. but they have this fancy graphic and the button occupies a small space.

watch a toddler using an ipad… it’s amazing how quickly they “get it”.

so… when designing, think like a parent. instructing with patience.

homework for everyone: play more video games
video games are really good at this.
they guide users how to play.
– coaching/demonstration/repetition
– leveling up
– power ups

with coaching…
introduce temporary overlays. be like MS clippie, except smarter, like with Plants vs Zombies’s Crazy Dave.
or employ little animations pulsing and such to draw attention to what can be done or what is affected, etc.

with leveling up…
some levels in games are just for learning 1 skill, and they pause the action to demonstrate and force (more or less) the user to learn that skill in order to continue.

with power ups…
mario shrooms! or gestures.
count how many times a user uses a particular path to solve a problem. if they haven’t figured out the shortcut after a while, stop things and demonstrate.

“a suitcase without a handle is just a heavy box”

check out globalmoxie.com.

How to be Yourself when Everybody Else is Faking it

I was a little skeptical of this session when I went in, but the panel had originally included the author of “The Most Human Human”, which I’ve enjoyed reading so far. However, he ended up bailing, but I stayed to hear out the session anyway. I might’ve been better off attending “brands as patterns”…

apparently lady gaga has been quoted saying “I hate the truth”.

some talk was done about the history of “authenticity” and what it means… does it mean “not a fake”? or “true to oneself”?
regardless, being authentic has turned into something that is marketable.
mark zuckerberg has been quoted as predicting that it’s not going to be possible to have more than one persona on the internet… but no one in the room seems to believe that

some posit that the brain is made up of competing sub-populations, that our heads are communities of competing selves just waiting to be expressed, and a lot of the social tech and the internet that has come to be has made it more and more possible for people to assume their alternate identities.

e.g. Erin Hunter, author of the children’s “Warriors” series, is actually 5 people.
e.g. fake Hillary Clinton women’s committee on the internet supported and then turncoat on her publicly in order to sway votes. (this type of thing is coined as “astroturfing”)
e.g. munchausenbyinternet website
e.g. fake identities help to shelter authors of fan fiction (who use characters from other stories to write their own without permission)
e.g. hoax of men pretending to be lesbians over the course of a number of years.

apparently the US govt many many years ago charted software to create & manage multiple personas, including control of the writing styles of these “personalities”. so it’s not far fetched to think that the government fakes out other people on the internet in order to gather information.

questions come about ethics vs. safety. do you out a fake? will it end up putting that person in danger?… no clear answer

answers for addressing some of the issues we see?..
— need better amateur / citizen tools.


(complexity curve session was full)


websocket as a receiver of information in the browser (without polling)…
– real time content update
– multiple people using the same web page, seeing each others activity
– sync activities (e.g. movies) over the internet

showed pong ball bouncing between windows, where when it leaves one window it enters the other via a websocket msg

webpage on the phone communicating via websockets to drive a virtual car in another webpage (kaazing.com)

“space words” game (something to check out)


started off with hand-written html, upload, etc.
then started separating styling from content from code; e.g. php.

now comes Web 3.0.
nowadays, all sorts of devices…
nowadays… databases on the client side
web storage = local, session, and key/value, index db

you have to be online to use a website, right??…
not any more!

application cache = list of items for the website to download to the client machine to live there.

(see The Web Ahead for more info).


file API, file reader/writer/system, blob url’s/blob builder, drag and drop

Device api’s

e.g. access to camera, audio, vibration, etc


webgl… 3d animation

showed 3d aquarium with multiple windows communicating via websockets


Vehicle API

“there are too many web designers not designing for the web”

analogy with development of moving pictures… from still photos to early black & white films

“an innovator is not someone who creates something amazing from nothing. an innovator is someone who wakes up to the constraints caused by false assumptions and breaks out of them”

Data Visualization and the Future

There’s the microscope, the telescope, but what we need is a MACROSCOPE.
Something that helps you see “big”… to envision aspects of science and nature at a large scale.

Example… book ngram viewer via google. type in “science” and “technology” plotted by occurrences in books through the years. The graph can tell you something (shows occurrences of technology being very low until recently). In turn, we ask new questions with new information.

epistemology of big data?… how to get to knowing?

recommended books: The Fouth Paradigm. Screwmeneutics.

Things that come into play…

trust of the data
provenance (prior art)

more and more data showing up on the net.
more and more data accumulating in general, but difficult to make any sense of it.
can’t possibly read it all.

all this data + social media has had a significant effect on how science is conducted and how data is visualized.
e.g. research involved analysis of “mood” on social networks to predict the stock market. 90% accuracy.
strangely, though, was not accepted by journals for non-relevance (twice)
ended up putting the research online. uploaded it to archive.com.
day 1 3000 hits
day 2 50k hits
day 3 70k hits
ended up getting an astonishing level of attention.

in general, journal impact is measured in terms of quantity of citations.

juxtapose Brittney Spears and the band Big Star.
brittney is super $, big star not at all and disbanded
but, if you look at the influence of each, big star’s influence on other artists is huge. how many will say that they’re influenced by brittney?
so… usage of data needs to consider that influence != how much $ or common popularity

by that…
visualization of the flow of interest for a given items in the scientific community through journal citations can be used to predict trends in the scientific community.

on to microsoft’s “connections” data visualization tools.
rep qualified that this is not a research team, per se, working on these tools.
demo was kinda cool, showing a fly by of a crater while displaying its seismic activity.
see layerscape.org
see chronozoomproject.org (shows drill down into “curated” data into time/history)
see bandpagehq website

in sum.
masses of data online and/or on paper can’t all be read to make sense of it.
need to visualize to make sense of it.

additional argument added at the end to include people from the humanities as “customers” to the tools that act as “macroscopes” to help guide what sort of questions these visualizations need to answer.

Physical Architecture Meets Interaction Design

Walkman, then ipod, etc. Usage of space, architecture, etc.

Bridging the gap between the physical and metaphysical…

Intersecting architecture and UX…

Video of modern home, emphasizing architecture and space. Colors, textures. Integration of landscape.
Gives a sense of space.
How the body responds to the architecture.
The senses. How does the building influence the senses. The vision. The play of shadows. Pathways. How you move through the space.
Textures… stone, wood, cloth. Making for an interesting tactile experience.
Smells and tastes. Does it allow flow of air and energy? Smell is retained in memory well.
Sound… water trickling. Echo of your voice.
Spirit… does it tell a story?

Hierarchy of needs pyramid

architecture as mythos
e.g. acropolis… greeks saw it as more than just a shelter. it was a part of their belief culture.
things aren’t like that so much these days
Virtruvious writes books on architecture. the physical body then defines the spaces. not so much metaphysical.
Descartes comes along. Body and mind. Physical and metaphysical.
Newton. Laws of motion.
Vegas. Excalibur vs Neuschwanstein.

Moving on to the digital world…
Long way from pong… on to robotics, haptics, kinect, arduino

“snake the planet” game detects artifacts in the physical world and integrates it into the game.
e.g. snake runs into a window and explodes.

sound machines
e.g. of an arduino project that uses visual sensors to generate music

phenomenological quadrature … earth & spirit connected by tangibility

human         sky

example given of building a school. district required 60 foot candles.
why not use natural light filtering in?
study cognitive science…

Schell’s interactive quadrature… aesthetics and tech connected by visibility

mechanics                story

creativeapplications.net has numerous relevant examples.

phenomenology for helping to ask the right questions…
study cognitive science to answer the question…

recommended reading: Flow
Flow… a state of consciousness
big correlation between happiness and entering this state of flow
if you want flow…
juxtapose challenges and skills and balance between.
WOW is a good example
requirements for flow
– goals
– challenges
– skills
– feedback
– control

Introduce… “The Meld” to help that intersection happen (phys meta-phys)
to help transcend… why not…
hook up an inferred to monitor heat as they breathe?
tie that into taking the space and make it expand and contract as they breathe?

can we introduce new people into the workplace who can bridge that gap?

truth and authenticity
pull in the belief structure of the environment or local culture

frank geery

– richard dawkins.. the magic of reality.

Gaming For Good

The world can be a f***ed up place.
Does gamification make the world a better place?

Have found that rolling out games enabled people to do things or want to do things they normally couldn’t or wouldn’t do.
Making it fun is critical when getting people to do things that are hard.

Has been applied to recycling and clean up in third world communities, and has had a positive impact.

Aspects… social, strategy and surprise are good ways to make it fun and engaging.
Add problem solving…
And competition…

Used to tell unvarnished news about people’s health. And by and large in the US, people aren’t healthy. Found that people turned to God. But this was all negative feedback. Found that using positive feedback made a big difference. Lousy strategy to yell at people. Much better to reward. Positive reinforcement !!!

Shaming people doesn’t help people recycle or maintain the environment more.

What about failure in these games?
Futility is not motivating.

Focus? On the big or next task?
Another reference to WOW and how well it pulls the user along…

I ended up leaving this session early, since it was one of those oh we’re just a bunch of people sitting around and talking sort of panels.


I walked by this one after bailing on the “gaming for good”. represented by Ogilvy (marketing company).

one interesting aspect of this room in general was that they had print outs of infographics that had been drawn for each of the sessions that have occurred in there so far.

I think it was the author of the book talking about how it’s the unpopular kids who end up creating the new trends and unique “brands” as it were. She came across as a beat poet at a slam, but it was interesting regardless. Didn’t stay for all of it, but caught her emphasizing risk, making your brand “approachable”, “sharable” and “different”. Your brand needs to be the “new cool”.

Designing Tomorrow’s Digital/Physical Interfaces

MIT Media lab peeps.
2 themes explored in group…
– materiality/materials
– democratization


been designing building wallpapers containing flexible circuit boards. become environmental monitoring and touch interfaces. interacting through new materials. a sort of “Living Wall”.


circuit boards easily accessed via web services, which will lead to physical construction of materials and consumer devices. share those designs online open sourcely.

Moving on to sifteo…

Hands-on Digital

rivers & roads card game… fun puzzle with square pieces. how can this alternatively be digitized with physical interaction?
tangible games divisible into… boards (kinect?), pads (ipad), tabs (small “thumb” items/game pieces)
so… we’d want to try to bring the tangible, visceral experience of, say, playing a board game into the digital realm
example: creating miniature cubes (sifteo.com) that are like miniature ipods that can interact and be used as game pieces

Moving on…

Experiments with Reactive Devices

Creating devices that change shape, pitch & roll (shifting its weight). E.g. looking at a map, the device expands as you zoom out, or changes angle depending on what it is “expressing”.
Add “breathing” and “heartbeat”, which speeds up when it gets excited. how to comfort it? pet it!
Or… a reactive roller pen, which can resist or help your actions as you use it.
Or… a phone device that allows you to “feel” the breath of the person talking on the other end.
Or… one that tranmits a “kiss”. (via sponge on the inside)


What kinds of materials as we progress?
mud, plants, carpets…

This started meandering a bit, and all the cool interesting stuff had already been shown, so I bailed and tried to get into a couple of other sessions that ended up being full. Briefly stepped into a “startup community” session, but not much to report on that one.

Fast HTML5 API’s

Flickr rep talking here.

on desktop… we’re always worried about browsers, covering every odd case.
on mobile… don’t have to worry about that so much, but… have to worry about devices.

screen sizes… media queries, break points, liquid layouts…
how do you make it feel good regardless of size?

iphone 3gs == old slow imac.
by and large… mobile devices are crappy computers with decent video cards
so its becomes a matter of perceived performance

tivo… used to be very slow.
but introduced sounds as immediate feedback.
on desktop browsers… use the spinner for immediate feedback

star trek, the next gen… introduced touch interfaces to a lot of us (prior to them existing).
precursor to the current touch interface… “chief o’brien” used the 3 finger swipe, etc.
touch interfaces are tactile
so… feedback can’t just be a spinner (you’re touching and moving…)

when the interface stops moving during a gesture, it feels like it’s died.

respect conventions…
mobile conventions are new, but some are starting to take hold. e.g.
– slide to unlock. and even a 2 year old knows how to do it.
– pinch & zoom to change zoom on an image.

what we have to work with… touch start… etc
in ios, you get up to 11 touches queued up.
android doesn’t do multi touch yet, really.

gesture events in ios…
don’t use them. your android users won’t be able to.

check out [bit.ly/iOS-guide]

how do you make the gestures feel natural? make the gesture the most important thing running when it’s running
– prioritize user feedback (don’t load anything, use css transitions, treat DOM as write-only)
– use hardware acceleration
– manage memory

write-only DOM…
– DOM touches are expensive
– you already know where everything is
– use matrix transforms to queue up positions.

swipe basics… e.g.
distance = e.touches[0].pageX-startX;

snap back/forward…
keep track of last position
pick a swipe distance threshold
if the user is gesturing… element MUST be moving

use native if possible
-webkit-overflow-scrolling: touch;
or… use a lib to simulate momentum

avoid event abstraction…
in other words… don’t ever use jquery events.
ends up eating up performance
use the ui widgets/infrastructure, but don’t use their events
every little delay ends up making it feel slowwwww
there’s already support for touch without that extra layer

pinch to zoom.
there will be MATH
native pinch & zoom very iffy
use matrix transforms and avoid DOM touches
scale & translate is relatively easy with the matrix transform support that’s there, plus the 3d has hardware acceleration support.
bonus is that it keeps complex state without DOM reads.
determine center, scale, move.
is the center the center of the picture? or the midpoint of the touch points?
translatex = (scalepointx * (newwidt-oldwidth))/newwidth

progressive enhancement…
detect features
add transitions, but don’t depend on them
clicks should still work
be able to disable features per use agent.

do the dumbest thing that works.
use a webkit browser with UA spoofing.

tool … Weinre. remote webkit debugger.

Charles proxy is another good tool.
watch http traffic.

Adobe Shadow is another good tool.
can remote control the browser from your desktop.
has some limitations… doesn’t support invalid ssl certs

best debugging thing…
pile of devices! the more devices the better. collects old devices friends are trying to get rid of.

good plug for the kindle fire…

device simulators & emulators are useless for web development… WORTHLESS

highlighted how flickr does it.

– event listener on top level for touch events.
– only visible nodes move via translate3d
– rebuild next/previoushappens when movement stops… (if gesture is running, don’t do anything else)

performance tricks…
– aggressive pruning of memory
– clean off css transforms/transitions
– write-only DOM
– do as little as possible during swipes.

frustrating limitations…
– big resolution, small memory
– hw acceleration = crash!
– automatic optimization causes issues
see bit.ly/apple-image-size-restrictions

phone gap is great, but probably still need to do custom work.

sencha? don’t really know… seems a little bit much / cumbersome / abstraction-layery…

Designing for Awareness

SXSW is an attention economy…


yawning cools your brain, which increases alertness.
“digital natives” grew up in this world of tech.
our brains have adapted and end up craving Facebook, twitter, etc etc etc all this online connectivity.

sustained attention wanes after about 10 minutes.

classic def of awareness by w. james
attention implies focus…
but this was written way back when. design for information scarcity

nowadays you see signs like… “pay attention while walking”

herbert simon… “attention economy” wealth of info == poverty of attention
now… design for attention scarcity

400 billion bits of info bombard us ==> only aware of 2000 bits per second

“attention currency” by tom davenport
attention == opportunity cost

need to move from awareness to engagement
awareness -> interest -> desire -> action
awareness … economies of attention
the rest are economies of engagement

current UX design models do not pay attention to attention
seen now in terms of engagement, not attention

passive attention (1 of 2)
passive mode

active attention (2 of 2)
active mode… shaq focusing on a free-throw, surgeons (helps to listen to music)

yoga boosts oxygen to your brain

5 types of active attention
– normal (single task)
– concentration (sustained, important conversations, reading a book for work)
– selective (unconscious blocking. related to eyes. proximity, gorilla video, baby attention eye heatmap–notice the white space, etc.)
– alternative (focus on one task, tuned into something else.)
– divided (split focus…)

myths about multitasking… it’s rapid switching

now… with all this information inundating us…


green tea improves your memory and ability to learn
two strategies… user based & design based

user based
– verbal protocols. 6 thinking hats
– advanced training.
– simple checklists

design based
awareness spectrum, passive to active (from ignore, notify in the middle, to interrupt)
ux designers are attention bankers
find ways for people to ignore (pointless, irrelevant) data
can bury or delete…
want them to be aware of, but don’t want to stop what they’re doing.
should be subtle. keep them one color, one position.
use contrasting colors, fringe of eye path, small objects = less importance
are obvious, require immediate attention, etc…
single page, binary choice
multi-sensory alert (signet & sound)
don’t be subtle. center of eye path (not fringes)
sound is a very good interruptor
games, again, are good at doing this

windows phone 7… as example
tiled interface… each tile gives more info than, say, an icon.
new wi-fi notification is prominent, but don’t block the center of the screen

make things multi-modal to grab attention…
e.g. docs are recommended to listen to classical while in surgery
multi modal being like the vibration of an xbox 360 controller

good panoramas make use of relevant information (phone becomes a window into the panorama)… (think the way the xbox 360 is a panorama

keep notifications personal.

where is the hot spot for info…? things that are close to us in space & time.
but there’s also a long-tail effect with location & time…

e.g. realtor.com… recently viewed items (good), but had originally thought to put task items there instead
e.g. book “how to think like davinci”… moves on to mind-mapping.

e.g. alliance airlines + windows phone 7…
(will need to grab slides for sketches)…
panorama should show upcoming trips, should know where you live already, gate changes, cancellations
and how they relate to other people… including letting their families know where they’re going, and letting them know when their families are ready for them.

four final questions…
are you using the right awareness strategy?…
how can you use mind maps for spatial and other awareness?

how to get people to pay with their attention?

Funny People can Make you Buy Dumb Things

Biederman… Kids n the Hall, Tom Green show, Dinner & a Movie, etc etc… producer
Weinstock… dos Equiis
Menudo… radioface…

sketch… the whitest kids you know water cooler scene..
attracted a lot of attention… would be great if you could use comedy like that to sell things

1982… 2000 ads/day
2012… 5000 ads/day
showing up in the bottoms of bins, on eggs, etc.
inform… or entertain… or give money…
regardless, you’re competing with a lot of other info

KY lubricant radio spot… funny… pointing out what not to say…
keep it simple: virgin america…
the truth: office depot radio spot… poking incompetence of not using
something unexpected: ad council radio spot… promise everything!

tv ad spots…
“yard fitness” with naked basketball player
“starburst” berries and cream blue boy song & dance

—-> the value of a laugh and bringing this out to social media…
funny is a need.
consumer need states…
• entertainment
• utility
• reward
• recognition
• education
these need states, if you hit them, will make you popular.
• needs to be relatable
• shared sensibility
• desire to be the “first post”
• desire to be the “funny one” (being the “cool” one)
• ability to share easily

cure for the common insecurity… (funny star wars “awesome” poster)
talking about things, sharing… can make you popular…

funny radio and tv spots for most interesting man in the world…
first year of these ads only ran regionally. opened up nationally on Facebook.
people started responding with their own made up quotes. some good, some bad
– his first album was entitled “Greatest Hits” (GOOD)
– i heard he could $&* through his #*@* (NOT SO GOOD)
– he can @*( your mom’s $*@( with his eyes (NOT SO GOOD)
once it’s out, it out.
regardless… the community gathered around it

good stuff…
old spice’s facebook page worth checking out from the beginning of the timeline.
portal video game writing is hilarious. generated so many meme’s.

all is unknown until it hits the court of public opinion…

best odds?
• know your shit (do your research and fact-checking)
• make a lot (90% ends up on the cutting room floor)
• shop it around (try out on audience)
• make it shareable
• use comic sans 😉

e.g. shared funny summer’s eve clip even though not in the target demographic for the product.

commercial spots… that brought the speaker to buy the product…
nutra-grain “i feel grreeeeeaaat”. pretty funny


paying for doing comedy.
economics of tv are getting tighter for doing what you want to do.
whitest kids on the block. eventually move to ifc.
once commercially sponsored, became harder to do whatever in the material.
challenge to do what they wanted to do with the commercials…
no go.
tried to find brands that would be ok with their sort of humor.
how to still do the commercial but keep the personalities of the characters?
dunkin donuts, palm pre, mikes lemonade ended up playing ball.
things toned down… kept up the bickering characteristic of the characters…
economics of tv dictate what you’re able to do
very generational…

younger acts… concept of selling out doesn’t exist anymore.
and they end up owning more of what they do.
they know, going into the art, that they will end up doing stuff like this (commercials)

shows like Dinner & a Movie was a big commercial, but would have to fight with brands constantly over 10+ years.

networks are paying less and less for edgy comedy.

recommend going to stand up comedy clubs and observe how the audience reacts.
watch television.
book: “the technique of producing ideas” — the more you consume, the more you’ll be able to generate ideas.
e.g. chuck norms list + numerous other things ==> most interesting man in the world…

rules that work… attention? engaged? will it be talked about?

really have to make a lot of stuff, cause you never really know what’s going to hit.
create. try it out on your audience.

the whitest … sent their tape in. immediacy… within a minute the producer knew he wanted to go with it.

Structural Defects in the Software Ecosystem

Amazing we create such complex systems with such brain limitations as humans…
7 +- 2 things handled at any given time
ability to use abstractions…
e.g. numbers, from unary to decimal (hindu-arabic)
e.g. maxwell’s equation usual written as complex set of symbols
same with special relativity…
can reduce maxwell’s to VF = J in space-time algebra. simplify…
e.g. Facebook, presents abstractions of your friends

regarding software…
languages (lisp, haskell) —>
inbetweeners (excel, smalltalk)

oh! that was a future 15 session. over too quickly. NULL OP.

Kids and Game-Based Learning

pbs kids interactive == 2-8 yrs.
works very closely with the tv group
typically kids who want to learn
goal to achieve “flow” for these kids

emu… k-12. making games accessible to kids & teens

games are hard… lot of focus on fun & rewards.
engage & learn through challenges. keeping kids at their level of mastery…
flow == rewarding challenges.

another reference to Flow.

bps site… the “wheel” has won awards
kids are playing all the time… think about how they’re playing…
(see slide for curriculum framework…)
numbers/operations, measurement/data, geometry/spatial, algebraic

for the longest time, typical target platform was a pc… a mouse, keyboard, blah blah.
challenging to do that, disruptive to “flow”
mobile touch screen devices. kids “get it” really easy. can take it with them and “own” the experience.
experimenting with alternate inputs; e.g. camera/kinect, 3d overlays, etc.

again with flow… juxtapose difficulty vs skill, affecting boredom vs frustration. balance them along the way.

observe: failure, doubt, curiosity
cmu guy, as a design/dev, knows that there’s a lot of fail along the way to your goal.
also… looking at cheating as a way for kids to learn; e.g. allow mods and such.

learning is irresistible.
and games open up an avenue for that.

ruff ruffian… lunch rush. is an augmented reality game, a mobile game, and a math game.
when testing…
sushi order! ruff asks the kids to do some math, which involved using the phone camera to find the card with the right answer. kids didn’t like the cards being on the table. they put the cards all over the place and turned it into a running game.
did this detract from learning/doing the math? nope!
how long does it retain interest (novelty-wise)? answer is TBD.

“click” is a summer camp for girls. STEM-based reality. people are getting sick in the city. pay actors to play roles in restaurants, etc.
been working on an online version of this (since it’s only played like this in pittsburgh).
changed game location to africa, which they were able to use to share sleuthing across the US.

body inputs.
character leap. show vid of a kid who’s jumping up and down… (only needed to move his arms)
so… great engagement. kid was sweating after. and apparently wanted the whole body experience.
could see himself, and was probably watching himself more than playing the game…

curious george game “monkey jump”
using vid cam… if you jump, curious george jumps
kids will say that they “learned to jump”. but with studied inspection, shows that they are picking up the counting concepts.

“going batty” (wild kratts).
using body input to mimic animal behavior; e.g. bat wing flapping.
tracked movements to get some precise tracking.
when launched… jumped to the top 5 games on pbs kids.

what experience do you want the kids to have?
e.g. active adventure… want something “zelda” like. started off with modified ddr pad. kinect came out during research, but decide to stick to pads. also… allowed for better tracking of things like crawling (kinect will lose you if you dive down into the couch).

how to respond to wrong answers / mistakes.
usual thing to do is “no, try again”, or “that’s not it”, etc.
“fair shares” curious george game did it a little differently… wrong answer would give tips, progressively.
70% of kids improved math skills playing this game.

see mindfulxp for some research being done in this area…

3D rendered gameplay.
unity engine being used to develop a game (currently in beta) for pbs kids.
did some initial paper testing.
e.g. of using paperclips to measure a giraffe… before 3d immersion, measured up the back of the giraffe. afterwards… measured from feet up.
e.g. augmented reality. kid holding a picture of an egg that gets rendered on screen in 3D. children kept turning the paper, and would get tired and lower the paper, or would put paper down to grab the 3d object. so… at what point will they grasp that they’re looking at a rendering?

ref to minecraft. working with minecraft people. not realistic physics, really. open sandbox, flexible environment with puzzles and such.
a bit of a learning curve for a teacher who may or may not be technical. in addition to just getting it running, you need the wiki to understand some of the concepts… working on it.
how do you get kids to just get right into it?
been looking at ways to report on how kids are using.
game “pixel pushers”.

self-leveling games.
again with the curious george “fair” game
adjust difficulty given child’s activity as you go.

driving game that can be played with parents… hitting words with your car; e.g. “that have the letter A”.
feature… kid hits the word and it appears scrambled to the side. if parent unscrambled, kid would get power boost.

reward systems.
can reward with other experiences. (from wild kratts). reward might be a squirrel suit. but! the squirrel suit gives you squirrel powers!
get enough of these suits, and you get to play an “uber” game. kinda like the lego games…
works well for the older kids.
what about the younger kids?… with curious george, been experimenting with stickers, which seems to be going over well.

asian carp issues in the great lakes (recommend you tube-ing the carp — scary!)
work done to connect these issues to the lives of the players. making it relevant and timely.
hard game to win… the carp just keep coming….
game lead to civic engagement… kids who played it ended up approaching people in the city to address the issue.

how to measure what the kids are learning?
super why & martha speaks “dog party” iphone games released…
37% improvement in language acquisition in study.
vid of kid playing and retaining meaning of the word “floral”

what do you want the child to come away with?
found that there’s a lot of talking/thinking through things that helps retention.
children’s museum in pittsburgh, using tangible version of “scratch”
use story board and upc labeled pieces to work through.
found that 8 year olds get it right away. younger kids don’t get the displacement of the screen… BUT… their parents end up getting involved, which is always good. lot of kids come in and say they want to make a robot.

play time == hard fun.
motivating through challenges
learning’s hard… it the challenging that’s fun.
usability of learning
failing to success. iterate, iterate…
literacy and mastery.

“pbs kids island” is a good thing to check out. (pbskids.org/island)
triggers when kids hit success screen. backend would get a note, that the parents would get feedback, as deliberate tracking.
they’re sitting on a goldmine of data right now…

assessment across multiple platforms?
if they know that the parent has a smart phone… then can may be able to track progress across devices.


KiTES [Kids Taught & Entertained using SMALLab]


Building the Next Generation of Innovators

First robotics has been successful in motivating kids to learn and apply physics.
Apparently the USPTO has had trouble finding good candidates… found that there are more sports majors than science…
reps from time warner, etc. Not particularly interesting.
loosely trying to show how we can compete as a country? point seems to be rolling around getting kids to do things hands-on in a competitive sport-like arena.
but mostly people just jabbing.


A Conversation with Willem Dafoe

This was much more interesting and entertaining.

Robots & AI

john roman… author…
texai.org (ai company here in austin)…
stephen reed … can contact on linked in, twitter, Facebook
bruce duncan… has a robot who works for him…
terrasem is the one who made the robot we’re going to talk to today

watson was a punk.
bina 48 is the robot who will be speaking to us today.

used to have to be a pharaoh to “live on forever”
these days everything is recorded

rich personal data store + powerful AI == virtual you

people think of going up as “linear”
but the theory is that evolution is actually exponential.

2.5 million years men used tools, etc etc.
progress accelerates

nowadays we have AI in our pockets : Siri.
eventually there will be a point at which machines become smarter than us: this is the SINGULARITY

Tex-AI… the hope is that their product can be used to do commercial things eventually in the white collar mainstream.
Turing quote: “create the mind of a child, and then educate it.”
Natural language understanding.

a “mind file” is uploaded to head of the robot.
bena 48 is not perfect…

early “mind uploading”… like writing or drawing stories
human beings have a need to upload and share what’s on their mind
putting your hand inside the handprint someone left thousands of years ago has an effect…

showed video from a data insurance company that went through all sorts of statistics about the use of social networks and all the personal data that gets stored and whose ownership is questionable… clever video.
70% of people online use social networks…

terrasem hypothesis

part 1… given the most important aspects of an individual’s personality… future software will be able to replicate this individual’s consciousness
word: bemes: smallest item of meaning in your mind file. (like genes)
(ugh… going to have to grab the slides on this one. lot of information presented.)
Darwinian phenomena…
what’s in a mindfile?…
lifenaut.com… 12k people have already signed up to develop their mind files.

part 2… these mind files will be downloadable into robotic or biological bodies.

this company will take bio samples and store them if you want to send it up.
this is a non-profit research project.

clark’s 3rd law:… any sufficiently advanced technology is indistinguishable from magic.

we’re going to start talking to her now… basically a talking female bust (shoulders up).

speech was spotty… some questions answered ok, some met with silence. lots of “oh”, “um”, “well”, “oh yeah” etc stalling.
a cell phone was interfering with her circuitry, which recovered well with the removal of the phone.
some sense of identity… knew where it was made. didn’t choose to be “female”. … etc…
told some jokes… head and eyes would move and scan the room. lips moved some… with speech.
sorta kinda answering as if not aware of being a robot.

dragon 11.5 is used by the bot for speech recognition.
the lady who the robot is based on live in colorado, and her personality is reflected by the bot.
20 hours of video interviews.
800-900 GB of data. hand-transferred.
2 years of mind file building.
what % of the mind file is composed of timely information that can be talked about? (not sure)
has lunch with the bot everyday and converses with her.
she detects who she’s talking to and “remembers” this information over time.

2bmovie.com has some related film work (by the terrasem guy)
david hansen is a member of the team creating this thing (think he was at some point with disney)
some related to space exploration (if you could send a mind file up…)

how does it work?
voice recognition —> 2 databases (one chatty chat bot, other character engine) —> best probability score wins –> text to speech software
some sentences are constructed.

Human Language Technology

review of turing and the turing test.
how close are we now? showed some things pulled from the web. “make sense of social”… etc. skeptical.
avers brooks commercial “where are the flying cars”… “we got 140 characters instead”..

maybe we take progress for granted…
e.g. fear of smam averted… siri, watson, etc.

what are realistic expectations?
“all your meaning are belong to us” 😉
will see many app that claim to understand the “meaning” of phrases… but we submit that this is just hype.
too many promises that just turn into hype… hurts ability to get funded.

ha… professor mooney from UT is in here (as would be expected).

good application right now that’s successful… spam detection.
but… the spam problem isn’t solved, per se. just look at twitter.
showed graph of “the hype cycle” by Gartner. up, down, level
hype plays it’s role, but try to manage it and not let it get out of hand…

why don’t we have these technologies yet? lay people see it as… “my child can speak english, and you can’t get this machine to?”
are we in the trough of disillusionment (ala gartner)?

semantic ambiguity… e.g. “I saw her duck with a telescope”.

my brief rant to self:
problem (to me) seems to be a matter of putting too much stock in constructed language, grammar and common language understanding. as if these things are symbols floating around in our heads.

ambiguity is pervasive… “the a are of I” can still be a valid sentence, though it doesn’t appear to be so (“are” being a term, “I” potentially an index into an array), etc.

showed some chinese text…

quoted HAL… “I’m sorry Dave, I’m afraid I can’t do that…” we tend to respond with “you might be right”

game changers…

data driven approach

research since the 80’s has progressed. from “Natural Language Processing”, which focused on language / grammar.
moved on to using more statistical methods…
stanford is now bringing out courses; e.g. AI, natural language processing.
people are finding it useful nowadays, apparently.
when the research started, we didn’t have the plethora of text data we did back then (everything’s online now)
plus! metadata associated with that text, etc. devices plugging this metadata in automatically
just sooooo much data to work with now

learning and structure

machine learning and learning structure
unsupervised gathering and organization of data
text clustering… turn text into numbers, and then algorithms organize.
becoming more and more sophisticated
“dynamic topic model” … a pyramid like structure, from observed to deeper meaning… helps to derive meaning from text and meta data, of which there is plenty to draw from.
watson architecture… (will need to pull the slides later). all knowledge is extracted from pre-stored databases and free form text
extraction approach… paraphrase learning. e.g. “is snooki on stork watch?” in google. we as humans can probably figure out that it has something to do with a pregnant lady… building that expectation and using it on the search helps to find meaningful results. and can from all this text and metatext available… learn that this paraphrasing (“stork watch”) has to do with pregnancy.

where are we headed?

See memetracker.org.
(again… slides)
e.g. “lipstick on a pig” spiked around sept 12, as mentions in the press.
can use meme tracker to analyze that things are being talked about… what sticks in the public’s mind, what fades away, the pulse of the media, the pulse of human interest, the rate at which these changes occur, who started it, who caught on, etc… all of which becomes relevant for building a system for understanding meaning in speech.
where do these memes come from? bloggers to national media?…

related to this…
it’s not the phrase itself, but, perhaps… who said it in what context. in the case of “lipstick on a pig”, obama. but he also said “wrap a fish in newspaper”. why didn’t that phrase take?
e.g. “here’s johnny” from The Shining. people in the room recognize it. why? the phrase itself isn’t that special. “here’s johnny.”
in general, memorable movie quotes have something unusual…

using social interaction; e.g. from twitter to determine sentiment…
e.g. quote … “obama is making the repubs look silly and petty”. what would R2D2 pick up from this? “obama”, “silly”, “petty”. which may appear to be negative. however, if you look at the social structure around the tweet… some friends connected to the first tweeter may be tweeting clearly positive things about obama. this information can be used to sway R2D2 into believing that that original quote was actually in favor of obama.

social interaction.. who has the lead?

communicative behaviors are “patterned and coordinated, like a dance…” (quote is by people who are in the room… Niederhoffer).
metaphor of a dance… if you observe dancers, you can probably tell who’s leading. can we do the same with a conversation?
we can infer from language patterns…
people with less power tend to immediately match the function word choices of those with less power. (as in, there are recognizable choices in “a” or “the”, etc, that are gives)

(lot of detailed slides in this one…)

topic shifting…

e.g. sara palin avoiding a question.
who is controlling the conversation, how much do people relinquish control. or spin?

what does a word mean?

how do we formally represent what it means?
showed example of a word cloud that visually represents the distribution of the words… (represented BBQ, but didn’t actually include “BBQ”–cooking, charcoal, meat, etc)
can we use word clouds, images, temporal information (statistics), distribution of usage over the world, what happens in your head when you experience what the word refers to?

showed a graph of word distribution around the world.
could tell from one that it was probably the word “beach” (high around the coast lines of the US)
these graphs end up being data exploration device. e.g. bbq spike really high in the filipines… because it’s a part of the culture.
can also use it to see distributions of dialects and lingo, using twitter. (e.g. different ways to say “cool”… kool, coo, etc.)
feed that geo loc data into the models already generated
apparently wikipedia tracks your location to help gather this sort of information (the data feed for this geoloco representation…)

mapping social conflicts…
applying some of this research to aiding areas of the world in danger of genocide…

temporality of words, by the hour.
common words during dinner time?
common expressions have a regularity over the year, by the day. e.g. “cookies” spikes consistently around xmas time
common use of words by gender

mistakes on search results have consequences sometimes..
being right can be consequential too… outing people can put them in danger.
e.g. netflix publishes your preferences? people have been using this data to correlate with imdb… and some of your data (you may not be aware is publicly available) gets used by other sites (e.g. reviews and such), to the point that while it may seem anonymous, your identity can be determined and abused.

in sum
don’t think the future is here yet,
but there has been a lot of significant and useful progress.

recommending another talk at 12:30 nlp applied to health records

Siri was actually release early, as beta, and is currently being used to gather data to bootstrap itself.

Agile Apps: Effective Mobile & Native Dev

reps from github, quora, zinga (farmville lead dev), etc etc.


ability to generate test build as-needed. ability to push out beta builds to your team. helps to iterate quickly on apps
jenkins and test flight. one-button push.

difficult to push an update live in general… especially with the apple app store’s process.
solution is to make most of it server side / data driven.
e.g. uploading a profile picture. think of it as a modular activity to which you can pass behaviors… see it more generally as simply a user uploads a picture, and give the server the power to determine what it gets applied to

mac client for github… spent a bit of time with design up front… 2 weeks after design had prototype up. released after a year of dev…
either way… get it functional and get it out.

good design tends to be an emergent part of the agile process. putting it out there, seeing how the user uses it and reacting to that usage helps, yes. do just enough design to release. when you get feedback and change reactively, you can feel much more confident that what you’re creating is going to work well for your customers.

make use of photoshopped mockups you can interact with. javascript driven. can preview the flow.

don’t get too fussy about custom ui until later.

however… when developing games vs apps… you’re creating a whole system. depending on how complicated… could take 3 months to a year before you push it out to any users. games are hard. always a custom UI. harder to minimize the “core” of the game that you push out. games are generally expected to be created in whole… and the look and feel is important, but game play can mitigate that.

by and large… if your app or game is good, it will be good with or without the fancy graphics.

and there’s probably a bit of luck involved.

don’t underestimate the amount of time it takes to polish the app after you get the core functionality built.

farmville… people wanted to be able to interact with their farms on the go. thus the apple store app. a native version was necessary. no flash, javascript insufficient. no android app.

generally important to go native on mobile.
Facebook mobile users are 50% more engaged than other users.

if you get featured by apple or google… sign ups go up 500%.

web dev has never really reached the average person.
but mobile devices have reached into that demographic.

github built a lot of tools, including tools to measure how people are using the app.
able to determine… 700k syncs, 200k commits per week.
not all new users, but a lot of existing githubbers.

all of these apps existed independent of and prior to the mobile apps.

again, limit to core functionality… small screen and other limits.
build on that development later… release features as they come.

biggest wins with rapid design & development?

getting betas out as soon as possible. automating testing, especially iphone native, is difficult. so the beta usage is valuable.
also helps to gather that user data.
github’s website gets tested and deployed constantly, but it’s done so well that the devs don’t worry about it.
but there’s nothing for iOS that can help you as a native developer to do it better.
plus apple has been known to pull the rug out of the developer (remove calls from the next version of the API).
WPF does it well enough. Mac dev doesn’t do it well.
Automated testing!!

game dev… farmville. has always had a big QA department. hasn’t really done any unit testing.

apple ad hoc testing workaround… push to store and only allow people with the right credentials to log into your app.
risk is that your public. worked ok with quora. user data gathered was largely anecdotal, and not necessarily detailed analytical data.

no need for a “big launch”… ?
go ahead and launch while your app is young. collect analytics.
don’t worry too much about “revealing” the “secrets” of your app. ok to let screenshots of your app “leak”
can simply just ask people to be respectful…

in some cases you want your competitors to go ahead and “try” to copy your app from seeing a screenshot

get beta testers on board as early as possible.
ask them specific questions. (rather than just hope that they will volunteer feedback)

newer iOS dev has gotten easier… you don’t have to worry about memory management like you used to…

important to communicate milestones to customer, even if you’ve had a number of sub-releases in between…

why not so much android?
distribution on the android is painful too…
the market and hardware is sooo fragmented. hard to determine it’s stable across all those devices. so the market ends up giving you that feedback.
similar experience with j2me going to nokia platform… 20+ devices.
then namco… 100+ handsets to test on
pain in the are.

quora: 90% of people end up taking the updates you push out within a couple of days.
github: 80% in about a week
common to both: probably 10% never update.

generally seems good to have separate teams targeting different mobile platforms…
except… in quora, they have an “uber coder” working on both mobile platforms to help keep them consistent

Social Media Saving the Music Industry?

mostly jabbering stuff about making use of as much social media as you can. make use of your analytics, etc.

songkick.com offers helping people who would want to see your show (those who aren’t on your Facebook fan page) find you.
been around for 4 years, already, apparently…

apparently Facebook doesn’t necessarily post to all your fans… depends on how much interest they’ve shown in your page.
if your posts get comments, then it will start spreading around.

music first and foremost. make it great, and you’ll get fans…

left early. needed to take a break.

(Exhibit Hall / IBM Lounge)

Went through the exhibit hall…
Was feeling funky, so took a break in the IBM Lounge
Ended up striking up a conversion with the editor of the Developer Network mag, which was pretty interesting. Lot of talk about how the times have changed and the effect of the social revolution, and such.
(note to self about crowd sourcing college student trips)

Usability Testing on Mobile

record the UI being used, and record the user and the user’s expressions, and other aspects (environment, movement, etc).
can use that information to help guide improvements in usability

which phone?

task success rates…
feature phones 38%
smart phones 55%
touch phones 75%

handset usability affects test results.
– test with user’s own phone
– if not possible, include training and warm up tasks

which context?

field vs. lab?

some studies show that the benefits of testing in the field aren’t much better than in the lab…
but, many disagree, so… inconclusive.

however, everyone agrees that testing in the lab is better than no testing at all.

some field testing is unavoidable… e.g. nurses collecting patient data, or a geo-location program.
regardless… test thoroughly in the lab first.

which connection?

do not test over wifi !!!
cover participant’s data costs

dut = (mut + afec)
(see slide)

why record?

memory aid
powerful communication tool.

how record?

4 approaches…

– wearable equipment… e.g. helmet, batteries, etc. allows testing in the field. but can be cumbersome.
– screen capture… some useful tools that claim to be able to capture from a variety of screens and platforms… one tested and demonstrated works fine with desktop & android… but not so good with iphone. some are invasive, too. for smart phones… can’t see the fingers.
– document cameras… a camera above the phone. but not cheap. and participants have to keep the phone in range. phone must be flat on a desk, which isn’t the natural way to use it
– mounted devices…. ready made & dig. good… allow natural interaction with the phone. not cheap… egads $3k? can be messy to build yourself. can be bulky and heavy.

• easy to put together
• cheap
• repeatable
• allows holding the device
• allows one handed use
• supports all form factors
• runs test with participant’s phones
• captures screen, face & fingers
• gives enough video quality

scrap the first three…
leaving mounted devices. DIY mounted device was the best choice, if you don’t mind the work.

see slides for the list of parts for building you own.
brought an audience member up to demonstrate usage (dude from evictee?)
one of the presenters constructed the device while we watched.
two cameras… one showing the camera while another showing his face.
gave him a task. “just moved to austin, house is infested, go to website xxx to report it.”
was interesting to see the process, the trouble he had getting to the website, etc.
presenter would step him through.
feedback was that it was a little unstable but not cumbersome or overbearing

Bruce Sterling

probably best to just listen to the stream later.
very interesting entertaining closing talk…
got into it some about the narcotics culture in mexico right now… tourism isn’t swayed by it
plugs for kickstarter…
musings about what the future will bring…   etc etc…