By Owen Fay . Posted on December 20, 2021

There are two main ways to optimize your website – before going live.

The first is to have two or more great designs, and the second is to choose the best converting one.

Usability A/B testing is a great way to get those additional conversions and make the right choice.

If you want to start usability testing, that’s great.

It’s an effective method to reduce risk and save time – which gives you the confidence to meet your website goals.

Want to make the right choices before going live?

Usability A/B testing is the best way to figure out which choice to go with – without suffering from individual bias.

Usability A/B testing is a human-centered process to evaluate a product, service, website, app, or other resources with real users to gain insights into their experiences. With usability A/B testing two variations of the resource are shown to a panel of users to find the option that delivers the biggest impact, achieves business goals, and drives metrics.

But before you start usability testing, you need to understand how it works.

I’ll explain how you can use this strategy to decide between multiple designs and thus reduce your risk of choosing the wrong one.

Just imagine: You have two designs given by the designer and you chose the wrong one due to your own bias. This can cause months of pain and re-work.

Wouldn’t it be nice if you could test the designs – before going live?

Let’s start with the basics.

How Usability A/B Testing Works

Usability A/B testing isn’t difficult to understand. You start with two designs (e.g. logos, ads, webpages) and then these two designs (A and B) get presented to a panel of users who choose the one they prefer.

Usually, when we have two designs, we ask a couple of friends for their opinion. That method is tiresome and does not yield significant results.

But what if you could ask a panel of hundreds or thousands of users?

A usability A/B test gives us just this capability. Your designs are presented to a panel of hundreds of users who vote on your designs.

Not only do they vote – but they also provide you with a detailed explanation. This gives you excellent customer feedback – even before you go live.

Here’s an example from our platform to give you a better understanding of what I’m talking about:

Once the responses are in, you go with the variation that won the test. You now have full confidence that a large audience agreed with your choice.

And you were able to do this in a few minutes – not days or months like traditional platforms.

Let’s begin.

Set a Clear Goal For Your Usability Testing Procedure

The first thing you need to do is determine which object to improve. Is it the logo? Is it the splash page? Is it the company tagline?

Don’t try and optimize every aspect of your website – all at once – before going live.

That’s an ineffective approach and won’t give you measurable results.

Instead, pick a specific object (like your logo or tagline) and then usability test it.

If you’re unsure where to start, here are some examples of different components you can usability test – before going live:

Usability A/B Test Your Logo

Your logo is your brand’s biggest digital asset. It is by far the most visible part of your website.

You probably created your logo using a logo contest or a site like or This would have given you tens of choices for the company logo.

The problem is: How do you select the best one for your company?

The key here is to narrow down the choices to 2-3 top choices. You can do this using your own judgment (which could be biased!) or ask a couple of friends.

But then you come to the final stretch where you have to go with the final logo.

That’s where you run a usability test and get the opinion of hundreds of people.

Pre-Live A/B Testing - Example Logo

This will not only give you a final choice, but it will also give you valuable consumer feedback about your logo and brand.

Usability Test Your Tagline

After your logo, your company’s tagline is probably the biggest effect on conversions.

You probably have multiple 8-12 word taglines listed out for your company.

The problem is: How do you select the best one for your company? Which message resonates with consumers?

The key here again is to narrow down the choices to 2-3 top choices. You can do this using your own judgment (which could be biased!) or ask a couple of friends.

Even better: Run a usability A/B test and get the opinion of hundreds of people.

Pre-Live A/B Testing - Example Tagline

You’ll receive the best option, and you’ll get valuable user feedback about any issues or potential improvements.

Usability A/B Test Your Splash Page

When visitors come to your home page, they give you about 3-5 seconds of attention. Getting the design and messaging right is critical for them to hang around and convert.

You have probably narrowed down your splash page to 1-2 themes (say from your hosting provider like Wix or ThemeForest)

The problem is: Which one do you go with? Which one resonates with consumers?

The solution: Run a usability test and get the opinion of hundreds of users.

Pre-Live A/B Testing - Example Splash

Have your designer create two versions of the splash page and input them into Poll the People and you’ll find the best version.

Usability A/B Test Your Landing Page

In digital marketing, a landing page is a standalone web page, created specifically for a marketing or advertising campaign. It’s where a visitor “lands” after they click on a link in an email or ads.

You have probably planned to put in tremendous time and money to drive traffic to the landing page.

So it makes sense to have multiple designs that would optimize conversions.

The problem is: How to narrow down the designs to a final choice?

The answer: Run a usability test and get the opinion of hundreds of people. 

Let the people give you feedback about key aspects of your landing page (e.g. “Which design is easier to navigate?” or “Which design is easier to understand?”)

Select a Platform to Run Your Usability A/B tests

Okay. You’ve got two designs and want to make a final choice.

But how do you implement the test – before going live?

Not everyone who operates a website is a computer engineer or programmer.

Fortunately, you don’t need to be either to run a usability A/B test.

There are plenty of tools and resources that can help you do that. But a lot of them take a lot of time and are expensive.

Poll the People does usability A/B testing a little differently. It’s quick to set up (less than 5 mins) and your test can be done in less than 60 mins.

If you want to optimize your time and money, it’s a great resource. Tests with 200 users costs less than $100 and you get all of the benefits of other platforms. 

Start with something simple:

Poll the People gives you detailed reports from your test and valuable consumer feedback.

You can easily analyze this information to make a choice and also refine your messaging.

It’s also free to sign up, which is always a bonus.

Setting Up The Test

The next step is to set up the usability A/B test. For our experiment, we’re going to use Poll the People as our testing platform but this will vary depending on what platform you use.

Don’t worry: The whole setup process is less than 5 mins.

First, we’re going to log in to and click “Logo Design” to get started.

Setup Pre-Live A/B Testing

Next, we’re going to set the test question. And upload our two images.

Keep it simple: Like “Which logo do you prefer for an electronics store?” or simply “Which logo do you prefer?”

Next, we’re going to set the number of respondents. For most tests, about 128 responses is a good number. This should give you sufficient confidence and get enough consumer feedback. Also, the test should complete quickly within about 60 mins.

If you need more confidence and less margin of error, you can increase the number of responses. If you are short of budget, you can choose a smaller number (like 32).

Set Number of Responses

That’s it. You are done. That was quick. Just checkout and your test will launch instantly.

Understand the data

Your usability A/B test will help you choose between two designs (A and B).

The people have voted and you now have the results.

How do you interpret them?

I’ll give you a quick refresher so you can effectively interpret and analyze the results.

The first thing to look at is: Who won? And by what margin?

Analyze Results

The greater the margin, the greater your confidence in your choice.

Then look at the feedback from the responses. Each respondent has to provide a clear explanation as to why they made that choice.

Analyze Responses

This gives you key insights into the minds of your consumers.


If you’ve read this far you can see that usability A/B testing with Poll the People has a lot of benefits and is a lot easier to do than you might have thought.

Optimizing your resources before they go live improves engagement, gives you ways to create great products and improves user experience, all while reducing risk, saving time and money.

Our goal with Poll the People was not to create just another testing tool but to create an affordable, easy-to-use tool that lets you optimize any part of our business.

As you test, you’ll see exactly what works right in your results dashboard. Now that you know the benefits of usability A/B testing and when to conduct your research, you should be good to go.

So, are you ready to make better business decisions? Go to Poll the People and create a project. If you run into any issues leave us a comment or reach out.

Owen Fay

Leave a Reply

Your email address will not be published. Required fields are marked *