Thoughts on User Testing
January 25, 2010 | 2 Comments
I’ve been moved to write this quick post in the wake of conducting my first usability test ever. I’ve always known logically that user testing is important, but today is the first time it’s become real to me.
I’ve been a busy boy…
I’ve been thinking about the UI for more than a year now and I’ve produced nearly two whole sketchbooks crammed full of ideas. Although I was tempted to delve in and produce a working interface in code, I wanted to follow best practice and do some testing on a paper prototype first.
I’ve spent the last few days refining ideas and putting together a well thought out interface. I made an interactive prototype with bits of paper and today I conducted my first test with the wonderful and extremely helpful Gavin Montague of Left Brained.
Iceburg, right ahead!
Within minutes of giving Gavin the first user task, it was clear to me that my design was, to not put too fine a point on it, a garbled mess. As the test continued, it bubbled up many assumptions I’d made that turned out to be invalid.
By the end I lost count of the times I thought “That was obvious to me. It’s not to him. That needs to be changed.”
Throughout the process, Gavin came up with lots of great suggestions of how I could improve interface elements to be much more usable.
It turns out that the mental model I’ve used and the concepts are pretty obtuse and suffer from extensive Cognitive Dissodance. Bottom line – rip it up and start again.
Assumption. The mother of all f**k ups.
When I started sketching the UI, I thought that the more time I spent thinking it through, the better. How could this be wrong?
What I didn’t realise that the longer you think about anything, the more comfortable with the concepts you get. That means that very quickly you lose your sense of what’s obvious to a newcomer.
The moment Gavin sat down and started using my prototype, I realised I only understood my UI because I’d been thinking about it for so long. If I could do this again, I’d be showing my ideas to people much earlier.
Release early and iterate.
I now understand why “release early and iterate” is so important. I was so close to building something in code and wasting huge amounts of time. And that kind of scares me.
Essential but hard.
I’m now left with this uncomfortable feeling of having all my interface foundations taken away and going back to the drawing board. It’s put me significantly behind time. And it’s hard to accept that I’m as capable as Microsoft of making a usability nightmare. Hmmm. Maybe not quite that capable.
However, with the constant feedback of users, I’ll eventually reach a usable interface. And when I finally start coding, I’ll know I’m building something my target market can use without tearing their hair out.
If I can do it, anyone can.
If you’re building a product, when did you last test your design on real users? If the answer is “not lately”, check out these resources and run some quick and dirty usability tests. You might be shocked at what it reveals. I was.
Comments
2 Responses to “Thoughts on User Testing”
Leave a Comment
Great post. And you’re so right.
Not so many years ago was doing web design in an agency where my process involved doing a quick sketch or two before diving straight into a high-fidelity photoshop comp.
I would always get worried about the point where I’d have to call my art director over to review my progress and endure some level of biting criticism of my design skills.
The problem with the situation was that after a few projects I was producing work with the sole objective of impressing the AD. Using visual techniques he favoured, etc.
Of course the reality was that the AD’s preferences were irrelevant, and the areas where we would focus attention would be of little concern to the eventual users of the interfaces.
Later I developed the knowledge that, as an individual designer, I couldn’t possibly understand how a user would interact with the pie-in-the-sky fancies we were building.
Fortunately I eventually got to the point in the organisation where I had the freedom to do some quick and dirty usability testing of my own. And I mean usability testing in the loosest possible terms… Basically just sitting an account director down in-front of an interface and watching him fail to succeed in completing even simple tasks.
As they say, the more you learn the more you understand how little you know. Many designers would take offence at someone questioning their design decision making, citing their education and experience as qualification enough. But I’ve learned that in order to achieve success it’s design research that should lead decision making – the skill of the designer is facilitating the use of techniques to uncover an understanding of the user. To an extent, letting the users design the interface themselves.
I think you made a great decision to sit someone down in-front of a prototype. How much would you say it cost to run the test? And how would you value what you learned in comparison to the cost?
I think rapid iterative prototyping is going to take off in a big way this year. In the last 3 months I’ve been involved in 5 prototyping projects, and all of them have been a fantastic success. I hope to finish a blog post soon about how I prototype with html/css/js, and how paper wireframes as a design tool have to die.
Awesome comment, Andy.
It sounds like you went through a tremendous learning experience.
How much did my test cost? Well, I “paid” Gavin with a packet of biscuits. And essentially I’ve saved probably 2-3 months of time that could have otherwise been completely wasted. That’s a pretty good ratio
“The more you learn the more you understand how little you know”.
Ain’t that the truth. Hopefully I’ll remember to design from the standpoint of complete ignorance from now on, realising that whatever I think I’m probably wrong.
I’ll be interested to hear your argument as to why paper wireframes should die and how this could apply to my project. I’ll keep an eye on your blog.
Thanks for a great comment and I hope to see you at TechMeetup tomorrow. I might be late, but I’ll try and get there.