Our usability testing lab setup

By Chris

Looking into the testing room

Moving into our new office gave us the opportunity to design and build the ideal usability testing lab. Combining the Flow team’s years of testing experience with my work as a sound engineer, we came up with a plan that incorporated everything we could ever want.

We have two ways for clients to view usability tests from outside the room. The first is from a room right next-door, looking through a one-way mirror. We have aptly called this room the Bat Cave as it is small, dark, and due to our fantastically unequal air-conditioning, always a bit cold.

This room is mainly for behind-the-glass testing, where a moderator can view and talk to the recipient through the one way mirror. This is great when testing live websites or very high fidelity prototypes. When a recipient is alone in the testing room they tend to be more willing to work out any problems may have with a site. We have found that it is very easy for a recipient to ask, “Am I doing it right?” when we are sitting next to them, but if we are in the other room giving them instructions every so often though a speaker, we tend to get more realistic results.

The room works well as a viewing space for small groups that can keep quiet, but as anyone that has watched a usability test will know, this is harder than it seems. Screams like “HOW COULD THEY NOT SEE THAT BUTTON?!” slip out by mistake and can usually be heard through the glass. So we set up the boardroom as the second viewing room. This is further away from the testing room so clients can go crazy with yells, comments and discussions without disturbing the test.

This led to our first technical decision: how to send the video and audio from the testing room to the boardroom. It could be done digitally over the network using something like Morae, or it could be done with a separate camera and mic in the testing room connected to a projector and speakers in the boardroom. We settled on the latter, mainly because of the flexibility. Along with the computer in the lab, the digital method would require another computer in the boardroom. It also would have made it difficult to stream the audio into the Bat Cave.

In the end we settled on:

  • A camera in the testing room sending video of the recipient’s face to a TV in the boardroom
  • A microphone in the testing room sending audio to the Bat Cave and boardroom
  • A VGA signal mirroring the screen of the testing computer to a screen in the Bat Cave and the projector in the boardroom.
  • A microphone in the Bat Cave to give occasional instructions to the respondent when we do behind-the-glass testing.

This way if you sat in either the Bat Cave or boardroom you where able to see and hear the recipient and see their screen.

With this analog setup we have not only been able to run website and app usability tests but also one-on-one interviews, mobile device and world-wide remote testing. All recorded and all viewed live the comfort of a boardroom. It gives us the flexibility to adapt the setup as we need it and as new testing challenges arise.