Michael Gaigg: Über UI/UX Design

12Oct2

Remote User Testing – A Comprehensive Guide

No doubt, user testing increases the usability and acceptance of your website and can/should be done as early as possible, preferably during Prototyping.

The following blog entry discusses the advantages & disadvantages of remote user testing, describes time estimates & costs and explains how a session looks like using Techsmith's UserVue.

General Discussion

Advantages

  • Reduced (or no) travel time and expenses.
  • Higher exposure through easy screen sharing (managers can sneak in easily).
  • Actual user environment, familiar and comfortable.
  • Possibly fewer drop-outs.

Disadvantages

  • Facilitator not physically present (degree of separation can be challenging).
  • Can't see facial expressions or non-verbal cues.
  • Difficult to build rapport and trust.
  • Difficult to control environment.
  • Possibly technical difficulties (firewall, etc.).
  • Setup and use of software or usability lab might be challenging and requires a liaison.

Time estimates & Costs

Task Breakdown

The following time estimates are to be taken with a grain of salt, they can change significantly up or down depending on the project size, experience of the team and infrastructure.

  • Preparation (18 hours)
    • User screening: 8 hours
    • Task creation: 8 hours
    • Environment: 2 hours
  • User testing (10 hours)
    • 2 hours per user
    • 5 users per round
  • Post-test (32 hours)
    • Test report: 16 hours
    • Implementation: 12 hours
    • Communication: 4 hours

Summary: Our Time

1 Round = 60 hours
2 Rounds = 110 hrs.
3 Rounds = 160 hrs.

+ User comps
+ Time additional observers

How it works

Software

UserVue

UserVue is a Remote user testing software that enables a Facilitator to remotely observe a Participant using a phone line for communication. Multiple Observers can passively join the Session and share their observations with the Facilitator.

Morae Manager

Morae Manager uses the collected data (observation markers and notes, video, keyboard and mouse inputs) to analyze and calculate task times, error quotes and other common measurements.

Session

A Session is initiated by the Facilitator. Invitation emails are being sent to the Participant and Observer(s).

At the announced time all the involved parties need to download a small software bundle that allows them to connect to the UserVue software. The Facilitator then calls the Participant and gives instructions on how to start the Session.

After the session has ended the installed software bundle will be removed from computers of the involved parties.

Participant Requirements

Operating System

Microsoft Windows 2000 and Windows XP or later version of Windows.

Web Browser

Internet Explorer 5.0 or later, Firefox 1.0 or later.

JavaScript

Enabling JavaScript is recommended.

Security?

All communication with the UserVue Web site is performed over an encrypted Secure Sockets Layer transport mechanism (HTTPS). All session data is encrypted with a 128-bit Blowfish cipher as it is sent over the network.

Can anyone "eavesdrop" on my session?

All session data (including audio, video, chat data, etc.) is encrypted with a 128-bit Blowfish cipher as it is sent over the network. This makes it exceedingly difficult for anyone to intercept and observe session data.

Are copies of my session data stored anywhere?

No copies of session data are stored on any server. The only recording happens directly on the facilitator's computer. Session data may pass through TechSmith's servers to facilitate firewall and NAT traversal. However, this data is never stored. Also, this data is undecipherable as it is in an encrypted form as it passes through TechSmith's servers.

What are your experiences with Remote User Testing?

References

Similar Posts:

Posted by Michael Gaigg

Comments (2) Trackbacks (1)
  1. WoW! Thank you very much for that enlightening article

  2. Interesting post. Thank you for sharing. You may like this article from Web Site Magazine that reviews usability tools http://bit.ly/32mqlQ

    I am affiliated with http://www.usertesting.com but the article covers several other tools as well (including Uservue).

    Amanda


Leave a comment