Why User Research is Invaluable
Two quotes from a recent New York Times piece on why user research in invaluable get to heart of why consumers and companies need to talk.
“I was intrigued by the fact that people in the microcontroller industry won’t actually be the people who invent what they’re used for,” Mr. Ford said. “If there is a guy who knows about microcontrollers and a pig farmer who knows about pigs, it will be the pig farmer who will see how to automate feeding his animals so he can sleep more.”
“I want to see how you get people to experiment. Maybe a washing machine repair man will figure out how to get the machines to report back to him and revolutionize the machines to get a competitive advantage. The point is that I don’t know what they’ll be used for.”
Quotes are from a researcher leading the efforts of ARM to put kits for microcontrollers in hands of everyday people. The project is named mbed.
This article inspires me to remind people that we know our hesitations when it comes to purchasing and sometimes, we the consumers, know how to solve the. User research brings this type of insight back into the organization for the different teams to use as fodder in their work – and after a few sets of iterations, we may see a product that does something useful for people.
From the New York Times, You Too Can Join the Internet Of Things, September 20, 2010.
How to Make Ethical Dilemmas into a New Assignment at Work
How groups of people work together has always been of my keener interests. Do they get amazing things done as a team? As a teenager working within animal rights and environmental groups, I saw how good intentions are not enough. I touched upon this in a past post, where I quoted Bill Buxton of Microsoft Research:
“A design managers product is the organization.”
Today I wanted to share a video. This 8-minute video produced by McKinsey, & Company “Voicing values in the workplace” makes a compelling, even if simple, point: don’t take an ethical issue and assume it’s an unlucky occurrence that’s a bit outside of normal work. Instead frame it as part of your job to deal with – a new assignment.
Professor Mary Gentile says taking the time to frame what’s happening in a way that works for you will be effective in helping you resolve the issue. This helps those unfortunates (no doubt, all of us at one time) to bypass the two typical responses- [1] suck-it-up and ignore it OR [2] fight it. Neither of those stances feels great. However, getting a new assignment that is part of your job, feels better, no?
It’s simple advice but I think subtle and clever. Here is the video:
Does this type of tip matter? Absolutely. Bad bosses will hurt a company. Here’s a recent example. According to PaidContent.org, Skype let go of a newly hired top executive after one month. Skype had second thoughts about this exectutive after he was attacked in comments left on a Techcrunch article announcing Skype had hired him away from Yahoo! I read over the comments. The commentators are relentless, and if they’re comments are generally true, I’m left wondering how all these Yahoo! employees individually handled their problems over the many years?
I agree with Professor Gentle’s point of view that resolving ethical issues as you come across them is an important part of your job and if they are not resolved, the performance of the company can suffer. The Techcrunch comments are worth reading.
Don’t Pass GoGo
Flying on a plane is as physically captivate as you ever will be in the world. An inflight Wi-Fi provider GoGo fails to take advantage of the this unique circumstance they find their (potential) customers in.
Here’s my experience followed by some thoughts on how GoGo could be smarter. (Despite the following critique, ‘ll say after a fast review of their website, the company seems like knows what it is doing – from design to pricing structures for its services.)
A few weeks ago, I was flying back from my native Boston after attending a close-friend’s wedding. A plane for me is a time to slow down, get away from the internet, and embrace the constrained quarters. I rather read a book or draw then watch a movie or surf the internet. However, on a flight back from Boston last week, after reading and drawing for awhile, I decided to give the Wi-Fi service a try out of curiosity, and pulled out my Mac from under the seat.
The Wi-Fi service was provided by GoGo. The signal showed strong and I connected quickly to their landing page.
They pricing for the flight at about $12 was reasonable enough that I was on the brink of paying it. But not quite. (I wonder if they’ve tested the elasticity of demand across different flights?) I still had my books and I just wanted to experiment with the service. I wanted to know, “Do they have a free trial?” I looked for the answer.
I didn’t find any mention of a free trial on their website. I went to the live chat, entered a fake name, and lined up as number seven in the queue.

GoGo chat session - I was 7th in the queue and waited eight minutes, which suprisingly I didn't mind.
It took me eight minutes to get to a person named Aaron. That would be deadly wait in a normal situation but in my seat on the plane I didn’t mind at all. Captivation is the unique circumstance of potential GoGo users.
When my turn came up, I asked Aaron if they had a free trial, and he wrote:
“I do apologize but at this time we’re not offering any free trials of the service.”
And that was it – the end of my relationship with GoGo. I closed my computer and put it back under my seat.
GoGo missed an opportunity to connect with me. So, how could the company do it better?
The first idea would be to give you free access for a limited period of time – say 5 minutes – and then give you more minutes if you complete various steps.
Here are a few ideas. Get more minutes if you:
- Confirm an email sent to your address. (The company must have a thoughtful engagement policy in this case.)
- Fill out a survey from the airline (about service, food, or whatever). They could be an additional service by delta. GoGo could create inflight metrics that it shares with the airline and the travelers.
- Friend GoGo on Facebook. (Viral: “I’m using GoGo on my flight from Boston.”)
- Watch an add. (I’m not wild about this one. It’s a classic way to give people more in exchange for their time, and ad dollars.)
My flatmate had a smart idea too – you could redeem frequent flyer minutes to get more minutes.
The main point is there is a lot of potential; you could think of even better ideas if you give some thought. The key is that GoGo owns a platform, which they need to use to capture more value and provide more service to people who fly sporadically. They shouldn’t have given me a binary decision – pay or go-away.
A Tale of Two Customer Satisfaction Surveys
A couple days ago I was browsing through my image files. I found two customer satisfaction surveys that I had captured with SnagIt – one from Zipcar, the other from Levi Strauss & Co.
Seeing them together was amusing – one horrendous and one wonderful. Zipcar’s one is a fantastic example of how a customer satisfaction survey should be done. Levi’s is an example of how a survey can contribute to dissatisfaction.
In truth, I’m loyal to both companies. Zipcar makes my life better and understands designing a service to make people happy. A long time ago, I worked selling Levi jeans to customers at their flagship store in San Francisco. I wear Levi’s event today, and I love I can walk in and buy 514s or 511s knowing they fit like I want them too.
This is a classic example of a how an old brand with a great history needs to be more aggressive in learning how modern companies use technologies and how to have a conversation with customers.
Levi’s survey should never found it’s way to customers. The survey either shows too many hands in the pot or at worst a company without a vision for the future, leaning on the crutches of bad data.
I’m not going to go into every aspect of the surveys but I typed out some thoughts about the differences between the two. You can click on the images to see the surveys.
Length – Long is Wrong
The quickest way to judge a survey is by its length – roughly, the number of questions asked. Long is wrong. Short is right. Zipcar has nine questions. Levi has 33, which means a person has to do 22 more clicks and more “cognitive work.” And Levi doesn’t even offer to buy you a coffee for your time.
The longer the survey, the more pressure to justify every single question. Also, I believe long surveys produce inaccurate data. Each successive question turns people into automation monkeys trying to get towards the goal of clicking submit if they don’t abandon all together the survey to eat a banana.
Form Design – The Organization of Questions
Zipcar has nine questions, but the last seven questions are grouped together into the third and final question. These last questions have the same rating scale. Redundancy is eliminated making these questions easier to fill out. Levi didn’t even try to group questions and they could have. For example, questions one through seven have the same rating scale. That’s needless repetition.
Net Promoter
Zipcar’s first question is the wonderful Net Promoter. They’ve implemented it in a great way too – a vertical scale instead of a horizontal one - which is easy for the user to scan. The rating text – to help standardize responses – is clear (1 = never, 5 = maybe, 10 = Absolutely!).
Levi also asks the Net Promoter, but buries it in question 22. By the time you get to it, you’ll want to lower your rating by a few notches.
Language and Confidence
Levi’s questions are needy and insecure. For example, they ask you to rate the quality of the images on the website and so on. It’s like the company is asking, “If you don’t like this, I’ll try something else. I did my best for someone with no vision.”
On the other hand, Zipcar is confident, asking straightforward questions you feel they have already thought about. They are just checking in with you, and not asking you to validate them.
Here is how they each set up their surveys. Zipcar entitled their survey “how’s our driving?” and the intro to the survey reads, “Are we headed in the right direction?” Ties into the brand, right? Absolutely.
Levi’s on the other hand starts out with the bland, “Customer Satisfaction Survey” and leads with “Thanks for visiting Levi’s.com.” Where is Levi in that? Nowhere!
Privacy Policy
Privacy policies are important for obvious reasons, but that doesn’t mean they should be a key element of every interaction with a user. It’s letting the legal team poke your customers in the eye. Zipcar has a privacy policy in their footer of the page and avoids sticking it in the survey. More importantly, in their brief intro, they ask you to just shoot them an email if you have any concerns – legal or whatever.
Your brand is your privacy policy. If your customers need to ask, they are unsure about your brand and they aren’t likely that loyal to you. When they ask to read the privacy policy, they are in affect stating, “I use your service, however, I keep it in the back of my head you might screw me because I don’t really trust you.” And when your company wants to put privacy policies everywhere, they are saying that they don’t trust themselves either.
Pointless Brand Dilution
Finally, Levi’s logo sits next to the survey company’s logo at the top of the questionnaire. I have no idea why Levi allows this. That’s poor form, either pointing to bad contractual negotiation or general lack of brand control. Either way, it shouldn’t happen, especially for a company that relies so heavily on its brand.
Meanwhile, Zipcar is zooming ahead, finding ways to please customers, as with their recent addition of FasTrak to Bay Area cars that prevents me from fumbling for dollar bills when crossing the Bay Bridge when driving north for a day of hiking.
Why Bill Buxton Doesn’t Report to a Programmer
Last weekend I was re-reading Sketching User Experiences by the Bill Buxton, who I admire for various reasons, two being he doesn’t mince words and he loves the outdoors. He seems widely loved too; I once met his Bloomberg Businessweek editor Helen Walters, and she gushed about him. So, I decided to see what content YouTube had up by him. I found a presentation he gave entitled “Design Thinking in the Wild” at Institute of Design Strategy Conference, May 2008. (The ‘wild’ being in corporations and in practice.)
Bill Buxton on sketching experiences, Institute of Design Strategy Conference, May 2008 from IIT Institute of Design on Vimeo.
His driving thesis was organizational design is the central task of companies that want to innovate and create great products.
He led in by saying most organization’s culture screws designers. In order to be affective, designers needs the right environment (and status).
Buxton drilled this thesis home from various angles. He used as a reference model construction+architecture to highlight how work ought to be divided up. The skinny: only 17% is construction, leaving tons of space for design. Design is about experimenting and thinking/prototyping out ideas. Time is needed for that.
He also said this: “A design managers product is the organization.” That’s beautiful. He said Steve Jobs gets it; he is a great design manager. An acquaintance, who works at Apple as an interface designer, concurs, “Apple gets it. Most organizations don’t. You have to put interaction designers in charge of stuff.”
Buxton dismissed the “designer genius” thesis, but also made it clear that every person is not a designer. He refuted Don Norman’s idea that everyone is a designer because they paint their house by quipping that this would mean everyone is a mathematician because they count change when leaving the grocery store.
You don’t need geniuses. You need high placement in the organizational structure. He highlighted something that I first heard mentioned in the documentary Objectified. Steve Jobs didn’t bring in new people when he was brought back to Apple. He worked with the same crew, which included Jonathan Ives, who had been at Apple since 1993. The implication is that Ives therefore had worked in an environment that didn’t help him. Then came Jobs, who empowered his team, because, as a good design manager, he got design. And the rest is history.
Buxton said IDEO is successful not so much because of the people but that when they get hired in, they get hired from the top, so they are situationally powerful from the start. He says the placement in an organization is the key prerequisite. Designers can’t work at the bottom of an organization.
Buxton joked that Microsoft had designers reporting to the programmers. This poke at his employeer made the audience laugh. Buxton said he had to speak honestly for people to take him seriously but of course the situation is better at Microsoft, and, of course, he doesn’t report to a programmer.
I’ve seen how placement in hierarchy works either for or against consultants we’ve work with. These consultants have ranged from the fantastic to the abysmal. The hapless ones can be “successful” despite themselves because of higher placement. And the fantastic ones, even though savvy with navigating politics, can’t do there job fully because they aren’t placed high enough. Consultants generally are bounded by the hiring boss. Instead of being a collective resource, they enter into a circumscribed space.
The larger point though is not about designers being placed higher and have more power up but about how the organization is designed, which he says is more important to get right than designing products. Organizations that are designed well will product great products.
One reason why is that good ideas come from diversity – a heterogeneous group. (This important point was first drilled home to me by Cass Sunstein in his book Designing Democracies.) Creativity doesn’t happen in on person’s neo-cortex. “That is so 5-minutes ago,” Buxton said. Good managers understand that ideas don’t happen in a brain. They happen by our social and cultural intelligences mixing and through the tools we use and the physical spaces we share. “It’s the diversity of people with many different experiences producing a lot of ideas of which you cut away at until you find The One. Design, after all, is “the most destructive profession. You start out with a million and end up with one.” You need the ability to throw out ideas – your own ideas.
And to do that as a designer, you need an organization that supports idea generation and idea destruction.
Design for Dystopia
I remember the first time I read books by authors such as Orhan Pamuk, Hermen Hesse, Milan Kundera, Graham Greene, Gabriel García Márquez, Albert Camus, Haruki Murakami, and many others. The pulled me into a new world and gave me new ways “to see.”
In order to “see”, you have to look at the darker side of life too. Dystopian novels specialize on dissecting that darker side.
The current issue of The New Yorker has a must-read piece by Laural Miller, “Fresh Hell: What’s behind the boom in dystopian fiction for young readers?” Miller reviews a handful of young-adult dystopian novels. Here a are few she mentions:
- The Hunger Games by Suzanne Collins where young children from different districts compete in gladiatorial matches to the death televised to the entire public.
- Uglies by Scott Westerfield where “all sixteen-year-olds undergo surgery to conform to a universal standard of prettiness determined by evolutionary biology.”
- The Maze Runner by James Dashn , where “teen-age boys awaken, all memories of their previous lives wiped clean, in a walled compound surrounded by monster-filled labyrinth.
The new rush of dystopian novels, although bleak, are not as dark as adult dystopian novels because they hold out hope for the young audience they written for – promising a “new, better way of life [that] can be assembled from the ruins.” Miller, says the books are taken by the young generation consuming as a reflection of current reality. “For young readers, dystopia isn’t a future to be averted; it’s a version of what’s already happening int he world they inhabit.”
I find it fascinating that a generation is gorging on these novels.
So, What can product designers learn from dystopia? Certainly, there is something around human behavior that the books help elucidate.
As the world gets washed over in social technology, the emergence of new social behavior provides a fertile ground for designers. If discerned early, the behaviors of a new generation can make a product delivered at the right time widely successful. And conversely, a product delivered at the wrong time will languish. Concerning technology, the rule of thumb has always been to launch a new product in the correct technology environment. If CPU’s are going to be 3x faster when you launch in 2 years, then build whatever you are doing for those speedy CPUs.
The same applies for behavioral environments. Yet there isn’t a Moore’s Law of behavior that I know of. Hence, the importance to hone your intuition about culture. There is a bleak side of life – where social pressures are brutal and human rights violations stream over YouTube.
Inspired by the dystopia, I’ve translated two classic deliverables into versions that are bleaker. It’s a first stab and rough, but here they are.
(1) Ugly personas and (2) Dangerous use cases
Ugly Personas: Personas, which are archetypical representations pulled from observing people tend to be rosy. Gina, the busy, professional mother who multi-tasks while raising two daughters. Bla-bla-blah. People aren’t always lovely.
It would be nice if a few personas were “ugly personas”that represent some archetyical awful customer – the dictator who buys the iPad. It’s harder to get at because people observed or talked to tend to be nicer. The nuances and little moments where awfulness live away from the spotlight and get missed by researchers.
Dangerous Use Cases:
Personas can feed into writing use cases, the scenarios of how you’re customers will interact with the product. The basic formula for a use a case is actor + system = goal. As with personas, use cases tend to be hopeful. A knife is used for cutting fruit not stabbing people. A cell phone is used for seamless communication, not a device that makes you run over people or send hate messages.
Dangerous use cases are where your product is used in negative ways.
It might seem odd to do suggest doing this. Shouldn’t product designers design for the best? Yes but it’s harder then ever to ignore the negative parts of humanity because the horrible things that happen across the globe are a click of a link away.
My thesis, then, is by assembling and tinkering with the ruins of the environment, you can build something better suited for it. Instead of basing optimistic design on optimism, base design on, say, the chaotic psyches of a teenage world.
What might emerge from an analysis that has some bleakness built in, might be surprising wonderful.
This piece is a stab at thinking this through. Comments welcome from anyone who comes across this post.
Fresh Eyes from Emerging Markets
In the last year, a few of what have developed into my most devoted friends, are from Asian countries. When I have a dilemma in my life, I float it around to friends, and increasingly I see how people from other cultures have fresh answers.
All this has made a point very clear: other cultures have solutions for your own world if you ask. So, I totally got it when I found through someone I was following on twitter about a design competition for emerging market citizens to solve first-world problems. The tag line is funny and poignant: The rest saving the West.
It’s embarrassing that such a competition seems novel. But it is, and it’s totally wonderful. As an American, I can’t enter. I don’t like that but I say enthusiastically, “Yes, help us. Everyone help everyone.”
But don’t help us with obvious things like the stupid obesity picture put on the competition’s website – that’s getting cliche! – but help us on more subtle day-to-day issues that we in the West assume have been solved.
As a side note: I realized, happily, that the definition of design thinking as floated by Tim Brown seems to be culturally independent.
On a related note. The April 15th edition of The Economist has a special report on innovation in emerging market.
Snippets of Information from The Real Time Web
Andreas Weigend, former Chief Scientist at Amazon, among other things.
Jan Pedersen, Chief Scientist for Core Search, Microsoft
Todd Levy, Co-founder and Head of Product and Engineering, Bit.ly
George Zachary, Partner, Charles River Ventures
Kevin Burton, Founder/CEO, Spinn3r
Here is an unsorted list of snippets of information from last night’s talk The Real Time Web : Imperative or Insanity?:
- 40k tweets per minute tweets AND also 40k bit.ly urls made per minute. The parity is interesting.
- George Zachary of Charles River Ventures has invested in both Twitter and Yammer (Twitter for enterprise)
- George Zachary will invest in squirrels if they have a good business idea. He said this (and it made the audience laugh including me) when he said he wasn’t sure why 95% of the entrepreneurs who approach him are male. VCs, he said, don’t have a bias in who they invest in if it makes sense as an investment.
- Hilary Mason, chief scientist at Bit.ly, has a blog post about lack of woman in technology.
- Systems competence (reliability of network) is a critical barrier to entry. You can steal ideas and you can steal the feature sets behind those ideas, but you can’t steal system competence nearly as easily.
- Charging for a service is a friction to the user. This is an obvious point, however, like many obvious points, it’s easily forgotten. Bit.ly decided not to charge regular people a minimal cost like $9.99. Instead, they’ll provide services for the enterprise and price accordingly.
- The real-time web is getting us back to more natural behaviors (conversations are synchronous, emailing are not)
- Posting on Facebook is neither personal nor public; it’s a third dimension. Example given was a person who posted on Facebook a list of reasons he was thankful for people over the past year.
- Avvo – reviews about lawyers has been sewed (only) twice.
- Emerging problem to tackle: websites for reviews of people.
- Tokyo Cabinet is a great database system for certain problems; downside is only the one guy who understands it completely speaks Japanese. Time for a robust community to form!
- Twitter is popular in part because of the “illusion of audience”. Why do I tweet? Because I think all of my 1000 followers are listening to it.
1/3, 1/3, 1/3 – HP’s Answer to Innovation
There is an interview with Prith Banerjee, director of HP Labs, in the recent McKinsey Quarterly about how HP structures its research to achieve innovation. Four highlights below.
#1: Balance Different Business Needs Using a Portfolio
Managing innovation through a portfolio approach took HP Labs two years, Mr Banerjee. That’s not bad for a large company. The point of the portfolio was to balance the needs of business units with the long-term growth of the company depends on innovation.
Research is divided into three areas, each getting 1/3 emphasis.
- Basic research that might “change the state-of-play.” This work is given five to fifteen years.
- Product research that is tied to an existing product and has a time line of 6 months to 18 months.
- Applied research blances the extremes of basic and applied research. This segment has business impact but has a longer time line of two to five years.
#2: Get Close to the End-user, who doesn’t live in America (“the next billion”)
HP has its seven labs around the world. The reason R&D is in India and not Palo Alto isn’t because of the cost of labor. The reason is to bring the researchers close to the end-user. Researchers need to be part of the context they are developing for. India has a billion people, 70% of whom have cell phones but only 5% of whom have a computer. Computers are comparatively expensive, Mr Banerjee says, yet people can afford comparatively expensive items (scooters, TVs) if the items have a perceived value. Computers are not perceived as valuable enough to own. Obviously, HP Labs would like to solve that problem and that’s why they have a location in India.
#3 Managers Must Create Incentives
Providing the right incentives and measuring the impact of R&D is difficult. Management’s task is to create incentives and not a list of goals from the top, which nobody listens too, Mr Banerjee said.
HP has a board in which researchers can pitch big, aspirational ideas. The board is composed of 1/3 lab directors, 1/3 technologists from across HP, and 1/3 business unit people. For an idea to get approved, the different groups have to align.
#4 Measuring impact with the $100 Test
Measuring impact of R&D doesn’t work if you try catalog every little item that got transferred to business units. Why? Too many little transfers get counted as successes. What to do? HP uses the $100 Test. Each CTO of a group is given $100. (I wonder if they actually get $100 for this game? ;) ) They are asked to allocate portions of the $100 to the technology transfers from R&D that had an impact. Mr Banerjee’s examples was something that allows HP to solve digital printing problem would get $30, while some incremental that improves a laser jet printer would get $1.
Basic summary. Innovation depends walking the line between long-term and short-term objectives while giving line employees incentives to bring their passions to work.
(After free registration, a video and transcript of the interview can be found at the article McKinsey Quarterly.)
The Twitter Feature is Entirely New
Twitter is beta testing with a small subset of partners a feature to let multiple people easily post on a company’s behalf. Blogs, such as ReadWriteWeb, have said that this feature already exists from third party companies:
“And the hot Contributor API is something that CoTweet has been doing for a while.” Link.

I think the feature is entirely new.
After reading Twitter’s blog post about Contributor, I realized the so-called already-in-existence hadn’t existed yet because Twitter has paid attention to user needs.
Here is the post by Twitter product team member Anamitra:
“… a tweet from @Twitter would include @Biz in the byline so that users know more about the real people behind organizations.” (My italics.)
Now, go to the Twitter pages of JetBlue or ZenDesk who use CoTweet (and also Hootsuite). You’ll see posts “from CoTweet” or “from Hootsuite.” 
Honestly, that’s really funny. It’s like me writing a letter to a friend on paper and signing off, “Miss you, Pen” or “See you in a week! Love, Pencil.”
The tool being used shouldn’t become more important than the person writing it. Anamitra wrote it this way:
“[Contributors] enables users to engage in more authentic conversations with businesses.”
The inclusion of “authentic” and “conversation” is central here.
Somebody smart at Twitter is paying attention to details and helping companies become more human by implementing their NEW Contributor feature.
Heuristic for categorizing features:
Group by how they satisfy user needs and not simply by other functional characteristics.





