- HOW TO: Get Started With Google Analytics (by Meghan Peters) - Helps you monitor and analyze visitors, traffic sources, content, goals, e-commerce,...
- Usability Testing: What You need to Know? (by hongkiat) - read about how to structure tasks and questions
- 30 Inspiring “About Me” Pages (by Stephanie Hamilton) - yes, get inspired!
- Designing Effective FAQ Pages (by Mohammad Moradi) - use them as a last resort, but most important: make them helpful and not require another FAQ page to decipher
I find the following list really helpful when planning and conducting user testing. I collect and refine it constantly and would greatly appreciate any comments or additions I have missed (and I'm sure I did).
- setup web meeting
- tell secretary to not delete account and associated recordings
- test connection, equipment and recording capabilities
- setup schedule for participants
- send connection info to stakeholders
- remind everybody to mute their phones (or whatever else is necessary)
- prepare necessary data and files
- hide windows toolbar
- close mail program
- enable full screen for all users
- show host cursors to all attendees
- allow access to observers
- share desktop
- clear user generated content from previous user
- reset application
- remove cookies
- start blank application (if that's part of the test)
- take a break/breather for yourself
- prepare your personal notes taking material
- get acquainted with name and capabilities of next participant
- provide water for participant
- start recording
- greet participant and get going
- clarify time line for test results (findings & analysis)
- send thank you emails to participants
- Using Power Structure and Gestalt for Visual Hierarchy (by Shawn Borsky) - ...success is more than just what you see: it’s in the planning, structure, approach, and execution.
- A Comprehensive Guide Inside Your (by Alexander Dawson) - talking about SEO... this is a fine summary useful for any website developer.
- 20+ New Apps and Websites for Designers Sep. 2010 (by Cameron Chapman) - is this really happening? Hard to keep up with all the good stuff...
- Picking the Right Tool for your Remote User Testing (by Matt Milosavljevic) - Matt mentions some really useful links in here.
- Helvetica Joins the Web Font Revolution (by Christina Warren) - get 2000 fonts and 25k page views for free - nice for most smaller projects.
- 10 Usability Tips Based on Research Studies (by Cameron Chapman) - essential read!! Go and check it out, will help you argue in your next design discussion!
No doubt, user testing increases the usability and acceptance of your website and can/should be done as early as possible, preferably during Prototyping.
The following blog entry discusses the advantages & disadvantages of remote user testing, describes time estimates & costs and explains how a session looks like using Techsmith's UserVue.
- Reduced (or no) travel time and expenses.
- Higher exposure through easy screen sharing (managers can sneak in easily).
- Actual user environment, familiar and comfortable.
- Possibly fewer drop-outs.
- Facilitator not physically present (degree of separation can be challenging).
- Can't see facial expressions or non-verbal cues.
- Difficult to build rapport and trust.
- Difficult to control environment.
- Possibly technical difficulties (firewall, etc.).
- Setup and use of software or usability lab might be challenging and requires a liaison.
Time estimates & Costs
The following time estimates are to be taken with a grain of salt, they can change significantly up or down depending on the project size, experience of the team and infrastructure.
- Preparation (18 hours)
- User screening: 8 hours
- Task creation: 8 hours
- Environment: 2 hours
- User testing (10 hours)
- 2 hours per user
- 5 users per round
- Post-test (32 hours)
- Test report: 16 hours
- Implementation: 12 hours
- Communication: 4 hours
Summary: Our Time
1 Round = 60 hours
2 Rounds = 110 hrs.
3 Rounds = 160 hrs.
+ User comps
+ Time additional observers
How it works
UserVue is a Remote user testing software that enables a Facilitator to remotely observe a Participant using a phone line for communication. Multiple Observers can passively join the Session and share their observations with the Facilitator.
Morae Manager uses the collected data (observation markers and notes, video, keyboard and mouse inputs) to analyze and calculate task times, error quotes and other common measurements.
A Session is initiated by the Facilitator. Invitation emails are being sent to the Participant and Observer(s).
At the announced time all the involved parties need to download a small software bundle that allows them to connect to the UserVue software. The Facilitator then calls the Participant and gives instructions on how to start the Session.
After the session has ended the installed software bundle will be removed from computers of the involved parties.
Microsoft Windows 2000 and Windows XP or later version of Windows.
Internet Explorer 5.0 or later, Firefox 1.0 or later.
All communication with the UserVue Web site is performed over an encrypted Secure Sockets Layer transport mechanism (HTTPS). All session data is encrypted with a 128-bit Blowfish cipher as it is sent over the network.
Can anyone "eavesdrop" on my session?
All session data (including audio, video, chat data, etc.) is encrypted with a 128-bit Blowfish cipher as it is sent over the network. This makes it exceedingly difficult for anyone to intercept and observe session data.
Are copies of my session data stored anywhere?
No copies of session data are stored on any server. The only recording happens directly on the facilitator's computer. Session data may pass through TechSmith's servers to facilitate firewall and NAT traversal. However, this data is never stored. Also, this data is undecipherable as it is in an encrypted form as it passes through TechSmith's servers.
What are your experiences with Remote User Testing?
- Bolt, N.; Guide to Remote Usability Testing; http://www.ok-cancel.com/archives/article/2006/07/guide-to-remote-usability-testing.html
- Gough, D., Phillips, H.; Remote Online Usability Testing: Why, How, and When to Use It; http://www.boxesandarrows.com/view/remote_online_usability_testing_why_how_and_when_to_use_it
This article was inspired by Integrating Prototyping Into Your Design Process - Using appropriate fidelity for the situation by Fred Beecher which I extend by the following:
Prototyping needs to be iterative throughout the project!
Goal of Prototyping
Prototyping is not only a design tool but a research and communication tool as well.
- It should assist in optimizing the main task (top tasks) and validating its/their efficiency.
- Furthermore this should not add cost to the project but reduce project expenses while increasing ROI.
So the goal is to use different levels of prototype fidelity to incrementally identify and enhance the user's task(s).
Ideally this happens linear (increase visual fidelity as you add functional fidelity) but typically it is bent to either side (see Figure 1) where more emphasize on
- visual fidelity can be beneficial for marketing purposes or
- functional fidelity can assist earlier user feedback trough user testing.
Integration into your project
Regardless of the project approach you take it will boil down into the fundamental project management phases of Requirements, Design, Implementation (and possibly others). Prototyping should not be solely perceived as a method useful during Design, it is essential during all 3 (or more) phases starting as early as Requirements phase.
I suggest the following approach:
Low-fidelity prototyping (paper / digital sketch)
- Create paper prototypes or digital sketches
- Design navigation architecture (workflow)
- Review with client
- User testing (optional)
- Iterate (until happy)
- Revise into 2
Medium-fidelity prototyping (simple HTML)
- Simple HTML prototyping (maybe even black and white)
- Proof basic workflow and important interactions
- Review with client
- Revise into 3
- High-fidelity prototyping (Enhanced HTML)
- Enhance HTML prototype (links and basic functionality)
- Settle on design (including corporate design, basic artwork)
- Review with client
- Revise into 4
- Start 'real' implementation
Each prototype (digital sketch, simple HTML, advanced HTML based on simple) should not take more than 40 hours of pure development (not calculating initial meetings and communication and possible variations based on project size) plus 80 hours reviews and iterating with client. Sounds impossible? Think twice. It is so much easier to modify a sketch than programming HTML. The 'real' implementation will be built upon a solid code foundation with a grown-up design already - voila!
Can I skip a prototype?
Yes, obviously you can. But it comes with a cost later on because you miss crucial information from the earlier phase and it is more expensive to implement modifications.
The argument I hear most often is that 'prototypes' are wasted time/money because they get trashed anyway. This is absolutely not true! Identifying problems early almost always saves money later on, you don't find anything out until you start showing it to people, enhancing the quality of the product will help money flow into your pocket once deployed and most important, prototypes don't necessarily need to and should not be trashed.
Low fidelity prototypes can be more than just ‘paper’, this could be digital wireframes that look like sketches, e.g. Microsoft offers software that tie sketches (SketchFlow) directly into UI design (Expression Blend) and subsequently into development (Visual Studio) - check out the WebsiteSpark Program for almost free licenses.
Don't bend too much!
Danger! Don't bend the curve from Figure 1 too much otherwise you end up with
- a highly functional 'prototype' but without design, i.e. without visual clues whether your client/users will like it (buy it) and without validation that you got your information architecture right OR
- a highly visual 'prototype' that looks sharp, sexy and slick but cannot be used and lack usability ("we just installed the app and now our users complain they [...]" - substitute the appropriate phrase for yourself
Creating medium- to high-fidelity prototypes can be considered proof-of-concept and can be beneficial to or sometimes even required by your project. Looking at Figure 1 that would mean to move their respective dots from Design/Implementation to an earlier phase.
What are your experiences?
Do you use / re-use multiple prototypes within your projects? Do your project structures support prototyping? To which extent?
This is the first in a series of blogs describing User-centered Design Methods. My goal is to summarize my experience, insights and findings across multiple literature and compile them into easy and quick to digest pieces for you to consume. I want to encourage you to comment your own experiences and give me feedback on why your company applies certain methods differently or not at all or something else altogether.
I personally don't like the term Usability too much, it's an empty buzz word. It means SOMETHING to everybody but isn't scientific enough to be taken serious. It's often interpreted wrongly and purely misunderstood by most. It's kinda like Psychology, we know it is important to understand fundamental human behavior, their problems and remedies, but I wouldn't pay a dime to go to a Psychologist. But who knows, just as Psychology got its scientific relevance and acknowledgment - partly maybe through the 'invention' of the IQ - hopefully Usability rises up to similar levels (Jeff Sauro offers interesting metrics via SUM (Single Usability Metric).
That's why I like the term User-centered Design. It works wonders with Project Managers and the-like, probably because Design is such an important term in their daily work. And when asked about Usability testing I can conveniently point out that this is only one tool of many in my UCD toolbox. But the really important sales trick is to know which UCD method is best used at what time in the project management cycle.
The following chart compares the most common user-centered design methods, outlines their cost and shows when to use them:
Overview of user-centered design methods
|Method||Cost||Output||Sample Size||When to Use|
|Competitive Studies||Medium||Stat. & Non-Stat.||5||Requirements|
|Paper Prototyping||Medium||Stat. & Non-Stat.||5||Design|
|User Testing||Medium||Stat. & Non-Stat.||5||Design & Evaluation|
|Surveys||Low||Statistical||20+||Requirements & Evaluation|
|Interviews||High||Non-Statistical||3-5||Requirements & Evaluation|
|Server Traffic Log Analysis||Low||Statistical||n/a||Evaluation|
|Search Log Analysis||Low||Statistical||n/a||Evaluation|
Not long ago, after having completed a full project management cycle (requirements, design, implementation and evaluation) the PM proudly announced to perform a Focus Group with his stakeholders. Showing the ready application, he thought, would surely impress them and lead to valuable feedback for the next milestone. This impulse isn't uncommon but has to be fought before it becomes reality. Does he really want to produce MORE and EXPENSIVE requirements? Because that's the output of Focus Groups. Wouldn't he be better off running 2 iterations of User Testing to reveal usability issues or a Survey to receive input from outside the development environment?
- Competitive Studies
- Field Studies
- Heuristic Evaluation
- Paper Prototyping
- User Testing
- Server Traffic Log Analysis
- Search Log Analysis
- User Testing
- The Usability effort is NOT proportional to the size of the project. Bigger projects spend less percentage on UCD with same effort. Regardless, as a rule of thumb assign 10% of the projects budget for UCD.
- Faster iterations of prototype design require less testers
- Fidgeon, T.; User-centered design (UCD) - 6 methods; Nov. 2005; http://www.webcredible.co.uk/user-friendly-resources/web-usability/user-centered-design.shtml
- IBM; User-Centered Design Principles; https://www-01.ibm.com/software/ucd/
- Nielsen, J.; Field Studies Done Right: Fast and Observational; 01/20/2002; http://www.useit.com/alertbox/20020120.html
- Nielsen, J.; How Big is the Difference Between Websites; 01/19/2004; http://www.useit.com/alertbox/20040119.html
- Usability in Practice: Three-Day Intensive Camp; Nielsen, J. et. al.; April 2006; Proceeding, Usability Week 2006