Guide to 5 second test UX questions

What is 5-second testing in UX? The five second test is a fast and effective UX method used to gauge first impressions of a design or webpage. Using a live webpage, high fidelity mock-up or prototype, or even a screenshot, you can expose confusing elements in a design or message and then revise as needed. 

First impressions are everything. The first 10 seconds of a website visit are critical for a business to clearly communicate their most important messages to an audience. Research by the Nielsen / Norman Group (2011) has shown that the average user stays on a webpage for 10-20 seconds before leaving. A study in The Journal of User Experience (2016) verified the validity of a 5 second test as “sufficient time to gather the user experience of an interface and quality of the design elements.” When visitors land on a page they immediately begin scrolling and scanning. They’re looking for answers and your page needs to let them know they’re in the right place. 

Running a 5-second test with a small but objective sample group can help improve your campaign landing page conversion rates, home page effectiveness, and product or service page value propositions. Being too close to the brand is a frequent reason for mistakes in design and messaging. Employees working on a project tend to focus on what they want to say rather than what visitors need or expect to see. This test helps you to get perspective beyond your brand and your personal opinions.

User testing recruitment and sample size

The gold standard for recruiting test subjects is someone who fits your ideal customer profile (ICP) and is unfamiliar with your brand, however recruiting that type of participant is expensive and time consuming. You can still gain important feedback from people who do know your company as long as they are objective observers who are not frequent visitors to your site. Interviewing friends, family, colleagues in non-marketing/sales/product departments (avoid senior leaders as well) and customers can yield insight on whether your design communicates the right message in a clear way. 

TIP: If you’re testing a mobile app/webpage experience look for a younger demographic or people who have accessed the site via mobile.  

How to recruit UX test subjects

• Ask visitors if they are interested in giving feedback as an optional element on your contact us forms. 
• Run a campaign through social channels (paid or organic) to recruit user feedback participants. 
• Run a pop up or slide-in message on your website with a link to the recruitment form (many visitors to blog posts do not know your brand and are visiting only because of the subject matter of the blog post).
• Include a link to the UX recruitment form in your newsletters and add it to the footer of your website. 
• Email past customers and ask them for 10 minutes to participate in a user feedback interview.
• If you use an NPS or customer satisfaction survey, you can add a field to gather more participants for future user feedback testing. 
• Use a paid service if you’re testing a full website redesign or you’re in the later stages of a design evolution. 

Many companies build a usability research database with demographic characteristics and track each contact with the person so that they don’t over touch the subject. Providing small compensation for their time (participants are entered to win a gift card in a sweepstakes or get a discount on future purchases) is often a good incentive.

How many test subjects do you need?

The five second test is a simple gut check impression. You should aim for 5-6 test subjects according to Norman / Nielsen Group. Ensure that you don’t over index on the subject background - a test with only friends or only co-workers will be biased. “As long as you’re including someone who is not too familiar with your site,” says Mark Inouye, Director of UX at Fluke. “People absorb your site easily. Best case is you recruit people with no familiarity.”

How to run a 5-second test

First, state the purpose of your test internally. Start with the problem. We want to solve for xyz. What do you want to get out of this interview? What questions do you have? What uncertainties are you trying to solve for? Clearly lay these out as a problem and hypothesis. For example:

• Problem: We are not hitting our conversion goal.
• Hypothesis: if we clarify the headline value prop and benefits layout along with the CTA, then we will increase conversions. 

Here are some questions your internal team may have regarding a design or layout:

• Are we targeting the right audience? 
• Is the copy confusing? 
• Are we using the right design element hierarchy such as size of headings or placement on the page? 
• Is the page formatted for scanning? 
• Do the icons convey clear meaning? 
• Is the primary image confusing or distracting? 
• Is the CTA button text clear or specific enough?
• Is there too much text?
• Is there too much visual noise?
• Are we addressing the problem(s) customers want to solve?
• Is the brand perceived as premium or trustworthy?
• Does the homepage heading communicate what our company does?

Guidelines

Flash the webpage or prototype up on a screen for five seconds, then cover the page. If you’re testing a live webpage, show only above the fold content, don’t overwhelm them with too much information. You can test in person or in an online meeting. 

• Let the participant know what’s going to happen. “Take a look at the design we’re going to show you, absorb what you see, and then we’ll ask you some follow up questions. We’re just looking for feedback, and there are no right or wrong answers.”
• Do not ask leading questions or yes/no questions, keep your questions open. 
• Do not ask more than 5 questions. 

TIP: Remember to emphasize that all feedback is valuable.

Here are some question examples for you to use when interviewing participants in your own tests:

Impression questions

• What do you remember from the page you just saw?

• What is this page about?

• What is the most prominent element on the page you just saw?

• What did you think of first?

• What words or images do you recall?

• Can you recall the company name or product name?

Purpose questions

• What is the purpose of this page?

• Who is this product for?

• What is the value of this product?

• What should I do on this page?

Feeling questions

• What is your impression of the design?

• What do you like best about the design?

• What do you like least about this design?

• What would you want to change about the design? 

• How does the design make you feel?

• Do you trust this company? 

• Does the company feel reputable?

Homepage questions

• What do you think this company does?

• What do you think this page wants you to do?

• What is this company’s value proposition? 

• Do you trust this company?

• What value does this company bring?

• Would you want to do business with this company?

• What did you see first? 

• What  thing that jumped out at you?

• What was the name of this company?

• What did you like about it? 

• What did you not like about it?

Service or product page questions

• What is the company offering?

• What do you think this service provides? 

• Who do you think it’s for?

• What perceived value does this product/service bring to someone?

Interpreting five second testing results

If your page is clear, your test subjects can easily recall the most important content and can identify the main purpose of the page. If the majority of participants understand your message and have a favorable impression of the design, then you can call the design successful and feel free to roll it out or prepare it for an A/B test against your original design. 

Campaign landing pages are great candidates for a 5-second test since you’ll get feedback from your campaign audience right away. If the majority is confused by the messaging or does not pick up on your preferred design hierarchy, then you need to make changes to the copy or design elements. 

It’s important not to bias your interpretation of results. Negative feedback is still actionable feedback. In your interview, make sure you record it and/or have a note taker so that you can collect the precise words used. Entering the feedback in a word cloud generator may help clarify which impressions are bubbling to the top. 

Next steps

Use the five second test as a first step for a new design before you A/B test. Mark Inouye also recommends using it at the tail end of the testing journey, “You can use it after you have a winning A/B test and you’re ready to implement the new design on a live site. Sometimes you lose some of the hierarchy or clarity in a revision. A 5-second test can confirm whether you’ve made the right changes.”

Additional resources

• A/B testing sample size calculator
• Confirming your B2B SaaS positioning and messaging