A guide to choosing the right user testing method

User testing involves observing people attempting to complete key tasks on your website. For example:

• Purchasing a product
• Researching a holiday
• Signing up for an account
• Finding a suitable course to study
• Getting in contact with a company

Watching users undertake tasks allows businesses to find out where their audiences may be experiencing confusion and any difficulties they are having. Findings from user testing give clear direction for design changes or A/B tests.

Companies and organisations can fix the problems that may be affecting your revenue or making users leave a website without engaging or converting. Observing real users interacting with your website is the most effective way to identify any problem areas and uncover opportunities for improvement. 

• Why run user testing?
• Benefits of running user testing
• User testing at Fresh Egg
• What types of user testing are there?
• When to use which type of user testing

User testing is a vital ingredient for creating and optimising your website – it tests how users react to your website, whether they can complete key tasks or goals and how they find the overall experience.

Adam Stafford, Co-founder

The goals of user testing are likely to vary from company to company. A lot will depend on what is being tested (for example a prototype or completed website) and what insight is required.

Broadly speaking, though the goals of user testing fall into three categories:

• Identifying usability problem areas
• Finding potential areas for improvement 
• Gaining a deeper understanding of the user’s behaviour and needs

User testing provides qualitative (and some quantitative) data around how the look, feel and structure of websites and platforms can be improved to better meet the requirements of the user. Essentially, it gives you the insights that you need to shape or validate your design ideas.

There are many benefits of running user testing on your website, including:

• Shows how users actually perform tasks, rather than how you assume they perform them
• Reveals unexpected differences in behaviour between user groups
• Identifies barriers that may prevent users from converting or interacting with a brand
• Allows a greater understanding of users’ mental models and their expectations from your website
• Uncovers which features or areas of a website should be prioritised when it comes to redesign
• Can be used as a way to solve design disputes as it provides more robust feedback than opinion-based design
• Brings the client closer to their customer

A better understanding of your users and how they interact with your website will help you to build a website that meets their needs.

User testing can take different forms, which we discuss in more detail in the next section. The steps we take to organise, run and analyse tests are as follows:

• Planning. We kick off the project by discussing what our client is trying to achieve, and then create a project plan around that to ensure that the research meets the requirements
• Recruiting. We find users who share the same needs, motivations and demographics as those of ‘real’ users
• Building the test. We create tasks or questions for those users based on the likely actions of real users; ensuring that they help us achieve the overall goal
• Testing. We run the research in whatever form is most appropriate – at all times making sure we don’t lead or mislead the users
• Analysis. We analyse the research in detail and report back with evidence-based recommendations for next steps
• Follow-up. We can run other services, including other forms of research, based on the findings of the project

It’s crucial that each step of the process is completed in order to ensure accurate, useful results at the end of the process.

User testing takes three main forms, all of which have different strengths and can be used to uncover a range of findings. They are:

• Remote unmoderated testing
• Remote moderated testing
• In-person moderated testing

All three involve participants attempting to complete key tasks on your website in order to help you identify areas of improvement.

Moderated testing, both in-person and remotely, can be more thorough and in-depth than their unmoderated test counterpart. The key difference is that moderated testing can give you a better idea of why users are doing what they’re doing. It gives you the chance to ask follow-up questions during the test that remove the guesswork from the process.

Read on for an overview of each type of testing, what they entail, when they should be used and how they are carried out.

Remote unmoderated testing involves users attempting to complete key tasks on your website from their own homes.

During remote unmoderated testing, users are given a list of tasks to complete in their own home in their own time with no moderator present. The output from these sessions are videos of the user attempting to complete the tasks while thinking aloud.

The main advantage of remote unmoderated testing is that it is normally much quicker to recruit participants and produce findings. Online testing companies have huge databases of users broken down by demographics who are ready to take part in tests almost immediately.

As you can select participants who match your target audience, remote testing will normally give you a more diverse range of users than if you are recruiting locally. 

With a large number of users waiting to start, you can run tests concurrently and outside of office hours. You can also include participants from around the world, without having to leave your office.

Remote unmoderated testing can be very cost-effective. Companies run bespoke online user tests for around £50 per participant, which is a small price to pay for the insights you get.

Remote unmoderated testing is best for you if… you’re looking for a quick way of getting some top-level insight into the usability of your website.

Remote moderated testing allows you to run user testing sessions with users no matter where in the world they are.

During remote moderated user testing, participants are taken through a list of tasks by an experienced facilitator. The sessions are run using audio and screen sharing software like Skype to allow two-way communication between people who are based in different locations. The output from these sessions are videos of the user attempting to complete the tasks while discussing their thoughts with the moderator.

The main advantage of remote moderated testing is that it allows you to run tests with users based anywhere without needing to spend precious time travelling to different locations. The only things users need to participate is a reliable internet connection and a computer or other device.

The moderated aspect of this form of testing allows for interaction with users. It allows the facilitator to engage with participants to find out more about their needs, giving you a richer source of insight and understanding.

The facilitator can really observe and interact with each participant, making them feel at ease and encourage them to act naturally. A good facilitator will avoid leading users in any way but react to different situations to get the most insight from each test. 

Remote moderated testing is best for you if… you’re looking to get deeper insights from your testing while engaging with users in different locations.

In-person moderated testing allows you to meet users in person and get deep and meaningful insights from sessions.

In-person moderated user testing is live and takes place in a test ‘lab’. A good lab is often a relaxed lounge-style environment where users feel at ease. As with remote moderated testing, participants are taken through a list of tasks by an experienced facilitator. The output from these sessions are videos of the user attempting to complete the tasks while discussing their thoughts with the moderator.

The main advantage of in-person moderated testing is that it allows the facilitator, and any observers, to pick up on non-verbal communication, which may not be as visible with other forms of testing. 

As the facilitator is in the same room as the test participant, they can build more rapport and put the user at ease. This helps to put the user in a more relaxed frame of mind allowing for a more ‘natural’ test to take place. The ability to engage more organically with users allows the facilitator to get the most value from each test and go beyond the basic usability of a website to find out more about the needs and motivations of each participant. 

Stakeholders should be encouraged to attend these sessions so they can observe the tests live.

In-person moderated testing is best for you if… you’re looking to get rich insights into your users’ needs and motivations as well as seeing how they interact with your website.

There are benefits and drawbacks to each of the different types of user testing. To decide which is right for you, it is important to consider what you’re trying to discover and what level of detail you need from your findings and insights. The following table gives a quick overview of the features of each user testing method:

Understanding what each type of testing is, and how it works, will help you to choose the right method for you.

The purpose of user testing is to gain insight into how people interact with and understand your website. User testing will help you highlight usability performance issues and can validate or debunk assumptions and ideas around website improvements.

Running user testing and acting on the findings from your sessions, will result in a better user experience and a better performing site.