The pros and cons of a year of remote user research

Up until March 2020, by default, in-person user research was our preferred way of undertaking activity to test new designs and uncover pain points with existing user journeys.

With testing taking place in dedicated user testing labs, client offices and our own office here at Fresh Egg, the preference was always to be in situ when conducting user testing. From warm offices to cramped box rooms and expansive aircraft hangers, there was never a shortage of locations to conduct in-person user testing.

Then COVID hit, and with much of the world placed into lockdown overnight, the world of user research had to react to the disruption.

As people switched online overnight, at any stage in the life of the internet has such an event enforced such a drastic change in such a short space of time. With the disruption, enforced home working, mass retail and leisure closures, the introduction of furlough and user behaviour changing due to new emergency draconian rules, it was more important than ever for businesses to understand how the changes affected user behaviour.

Supermarkets failed to keep up with online demand; businesses had to move online overnight. Even the mighty Amazon creaked at the seams with the sudden increase in demand overnight.

With online means being the sole method of generating income, understanding user behaviour became more crucial than ever. In adversity, in a global pandemic - with people and businesses flooding online - a new golden age of the internet had emerged.

But, with this, a challenge, in-person moderated using testing could no longer occur, yet the world of online user research had never been more critical. For charities now starved of traditional donation means, having optimal donation funnels was no longer nice to have; it was essential! For eCommerce vendors, friction in the checkout experience would be even more costly. Having the ability to perform user research and testing became necessary.

There are three primary ways in which to conduct user testing, these being: 

• In-person moderated testing
• Remote unmoderated testing
• Remote moderated testing

In the early days of conversion services here at Fresh Egg, remote unmoderated testing was our primary method in which we conducted user testing. As our service offering matured, we expanded how we could assist or clients with the service.

Before the pandemic hit, most of our user testing had progressed to be in-person moderated. From March 2020, to provide our clients with the insight they require, this changed to remote moderated. The transition to remote testing was undoubtedly made easier by the overnight mass adoption of technologies like Zoom - especially as Zoom had become part of everyday life for so many people.

Of course, swapping to remote-only user testing does not come without its challenges, for example, the following all posed problems:

• Technical challenges are the obvious biggest hurdle (poor connection etc.)
• Giving access to restricted prototypes
• For example, the need to set IP exclusions for testing prototypes
• It can be harder to build a rapport and put users at ease when not in the same room
• Distractions for the user can be an issue (children, inbound calls, pets etc.)
• Audio needing to be clear for everyone
• Mobile screen sharing could be a challenge for those not used to sharing their screen

There are numerous advantages of running research remotely, including:

• No travel time, which is better for the planet, participants time and transport costs
• While distractions caused issues, they also presented a more natural environment - that is, when people are undertaking their journeys online, it is natural that interruptions will happen - this does not typically occur in a controlled user testing environment
• Better accessibility for participants with access needs
• A better range of the country covered, i.e., we're not restricted to London/Worthing/Brighton
• As a result of better coverage, the selection choice of candidates is often broader
• It's more of an authentic experience, particularly for those who work from home and using video calls to share their screens

While most user research is a form of user testing, other valuable forms of research can occur, such as card sorting exercises, focus groups and stakeholder interviews. User research methods that we use successfully in a remote environment include:

• Desk research
• Stakeholder interviews
• User testing
• User surveys
• Tree testing
• Heatmap analysis
• Immersive research
• User interviews
• Focus groups
• Card sorting

While the last year has presented challenges, there isn't much that we cannot achieve to the same quality as in-person testing.

Like with real-life slowing waking from the nightmare situation we have endured in the last 14 months, we will likely see changes, but not a total return to the way things were. While user testing labs are (slowly) opening again, the need depends on participants and their appetite to enter into a 121 scenario again. The pandemic has shown us that we can work from home, have a better work/life balance, and lead to a more varied mix of user testing.

In my opinion, we will see a hybrid. What we do know is that we can work remotely and get excellent results. That said, I think there is no substitute for being in the presence of people and would like to think there will be occasions when we are back in a test lab environment.

I believe the hybrid approach to user testing manifests itself very much like many workplaces will be more flexible in welcoming back people post-pandemic.

The time during the pandemic has given much greater flexibility for future work, both in moderated sessions and workshops.

While our methodology has not changed, the flexibility in delivery has. We are helping more clients than ever with their user testing needs as more organisations adopt an audience-first approach. Of course, we will continue to develop our delivery as the demands and needs of the user dictate. Still, the outcomes will be the same - actionable insights that improve the user journey.