Slide 1
Usability testing
Effective user research to make the web more usable for more people
Slide 2
Slide 3
This isn't a talk about tools, technologies or frameworks
Slide 4
Slide 5
My satisfaction
Satisfaction comes from… Knowing what we create satisfies and solves problems for users
Slide 6
Making too many assumptions
- Our users have the latest devices/software
- Our users don’t have any disabilities
- Our users should already know how to do that
- Our product isn’t that widely used to matter
- Previous products/approach worked fine
Slide 7
- We build features people don’t need or hardly ever use
Outcomes of not doing usability testing
- We may fail to consider different use cases, challenges/constraints
- Potentially break the law, get sued, bad press and reputation
- We turn people away to alternatives/the competition
Slide 8
Usability testing / research
Slide 9
We’re testing the website/product. Not the user
Slide 10
Quote from Just Enough Research
“The goal is to determine to what extent the product or service as designed is usable”
Erika Hall Just Enough Research | A Book Apart
Slide 11
Slide 12
Slide 13
Learnability
Do I need to consult documentation to use your website?
Slide 14
Efficiency
I have a 100-odd other tasks to do today. Will this take long?
Slide 15
Memorability
This website isn’t how I remember it. How do I do that again?
Slide 16
Errors
Users struggle to find X resource. How can we make X easier to locate?
Slide 17
Satisfaction
Is the website accessible and somewhere you’d happily visit again?
Slide 18
Methods of user testing
- Explorative
- Comparative
- Thinking aloud
- Remote
- Hallway
- Assessment
- Review-based
Consider one or more different approaches to user testing
Slide 19
Explorative
- Ideally performed by experienced testers
- Evaluate new or initial design ideas
- Iterative tests, defines future tests
- Test sessions are defined by goals
Slide 20
Comparative
- Two or more design variations
- Rate positives/negatives in user experience
- Typically A/B testing
- Combine best features
Slide 21
Thinking aloud
- Typically a series of pre-defined tasks
- Participant describes what they’re doing
- Gather feedback as they progress (or struggle) to complete the task
Slide 22
Remote
- Synchronous testing Real-time user testing via video link or remote sharing tools
- Asynchronous testing Automated/prescribed tests the user may choose to do in their own time and in their own environment
Slide 23
Slide 24
Hallway
- Testing done in high footfall public places
- Voluntary participation random individuals
- Quick, low-cost way to get feedback
Slide 25
Assessment
- Testing of a prototype ahead of production
- Real-time monitoring often from another room
- Progress and reactions to series of tests
Slide 26
Review-based
- Typically tested by experienced testers
- Pre-defined tests, often using automated tools
- Deep interrogation tests pre-production
Slide 27
Rating the problems
Rate each problem users encountered during the tests with a common scoring system
Severity
- Fails to complete task (High)
- Completes with difficulty (Moderate)
- Minor problem (Low)
Frequency
- 30% of participants (High)
- 11-29% (Moderate)
- 10% (Low)
Slide 28
Slide 29
Find the right participants and treat them well
Slide 30
Aim for 5-10 participants and usability tests
Slide 31
Avoid using your own team as participants
Slide 32
Testing for accessibility
- Get representative users
- Choose participants with a range of different disabilities and combinations of disabilities
- Allow users to test with their own aids such as screen readers, devices or other assistive tech
- If required to attend in-person, check the venue and transport to there is accessible
Slide 33
Testing environment
- Go to where the people are
- Have separate rooms for the participants and observers with good quality mics and network connections
- Choose a comfortable space: a relaxed, home-like environment away from distractions and lab-like conditions
- Encourage honest feedback
Slide 34
What to ask
- Get to know your participants and build a rapport
- Profile them and their web usage habits
- Have scenarios to work through
- How would you do…?
- What do you make of this?
- What would you do here?
Slide 35
Slide 36
Handling feedback
- Choose a good facilitator
- Friendly but neutral
- Encourage honest feedback
- Acknowledge problems/failures positively
- Be confidential and protect user data
Slide 37
Guerrilla User Research
“help the participant visualise the future product clearly enough to give useful feedback on whether they can achieve their goals”
Jamie Levy Chapter 8 | UX Strategy | O’Reilly
Slide 38
Guerrilla User Research
- Validated research conducted in short space of time on a small budget
- Small, structured experiments with a prototype and 5-10 participants
- Typically done at a coffee shop with good WiFi, power sockets, not too busy and acceptable to stay for the day
- 1 participant, 1 interviewer, 1 note taker
Slide 39
Slide 40
What usability testing isn’t going to do
- Provide a breakthrough design
- Guarantee a successful product in the marketplace
- Prepare you for every single scenario or use case
- Substitute for QA testing (but can help define QA tests)
Slide 41
Some typical feedback from usability testing
- Difficult to use and overwhelming forms
- Links and resources aren’t highlighted
- Challenging/hidden navigation menus
- Content isn’t presented or tailored for different audiences
Slide 42
Be open about your usability testing. Share your findings
Slide 43
“approach usability testing with the right expectations and conduct it early and often”
Erika Hall Just Enough Research | A Book Apart
Slide 44
Just Enough Research by Erika Hall
Slide 45
UX Strategy by Jaime Levy
Slide 46
