How to Conduct Concept Tests Quickly & Cheaply

Not only can you use Verify for concept testing, but it also works reasonably well for quick unmoderated usability tests.

Update: Sadly, Verify no longer exits.

For a total of $1,594 ($1,470 for 294 Amazon.com gift cards + $75 for SurveyGizmo + $49 for Verify) I was able to learn several surprising results in a recent landing page concept study. For example, the 2 concepts that I thought for sure would win, didn’t. And the best practice of putting the lead gen form on the right? It performed the worst.

Case Study: Concept Testing a Landing Page

One of my clients asked me to ideate, then create different versions of their most successful landing page to see if we could achieve even better results.

A/B testing the landing page — even a small sample size — wasn’t possible for a couple reasons. Without data to support the changes, our proposed ideas might be seen as too risky by the leadership team, plus an A/B test required development resources that wouldn’t be available to us for awhile anyway.

So we did the next best thing: we concept tested mock-ups.

For the study, I was given a budget of $2,000-$3,000, so I knew right away that any research tool that charged per participant was out of scope. $49 per participant? Not on my budget.

Our study included 8 landing page variations (plus the control) and ~300 total participants. I intercepted people directly from my client’s website using LP Marketer (a LivePerson tool), then used SurveyGizmo to ensure participants fit the target profile. From SurveyGizmo, I was able to link qualified respondents directly to the concept tests in Verify.

Sample Test Question from VerifyApp.com

Verify Is Fast & Cheap, But Not 100% Awesome

Verify (by Zurb) is a scrappy testing tool for those on a tight budget, but there are a few limitations and pain points to consider.

Verify Pros

• Super cheap (plans start at just $19 per month)
• Unlimited participants at any plan
• Unlimited tests at any plan
• Unlimited users at any plan
• Multiple types of tests and built-in test templates
• Ability to link multiple types of tests together
• Click tests (see where people click!)
• The tests can be taken on desktop, tablet or mobile

Verify Cons

• Cannot make fields or questions required (participants can easily skip questions)
• No restrictions to keep participants from taking the same test multiple times
• Little to no ability to customize test questions (some questions can be customized, some cannot)
• Gratuity is handled separately (email address cannot be made a required field, so several generous participants didn’t get paid their gratuity — yuck)
• No ability to integrate a screener (participants can easily share your Verify test with people who aren’t your target audience)
• Reporting capabilities are pretty much 100% manual (there are no date/time stamps, plus linked tests need to be manually merged)
• Interface is not super intuitive for participants (there is a learning curve, for sure)
• No ability to set quotas (refresh test results often!)
• No randomization (as a work-around, I used SurveyGizmo to route participants to different Verify tests)

Sample Click Test Report from VerifyApp.com

7 Tips for Using Verify & Other Things I Learned the Hard Way

1. Choose “linked test” first, then you can customize which test formats you want to use (e.g., preference tests, click tests, memory tests). You can also add tests or reorder tests within the test flow, if you make any mistakes. Use drag and drop to reorder tests.
2. Important: Create a “throw-away” test as the first question or task in your study. It’s not super intuitive for participants to understand how to use the tool at first. Set up the first test to be super obvious and simple, so that participants can “practice” and become comfortable with the flow of the tool.
3. Optimize your mock-ups for the web, otherwise they can take several minutes to upload into the tool. Also, size all the concepts in the same way, so the screen sizes don’t jump around from test to test.
4. Most participants won’t know they can scroll down the page, so try to have your concepts or mock-ups fit above the fold. I only know this because I ran some in-person practice tests, and saw that few used their scroll bar. (In addition, most of my practice testers — tech-savvy people, I might add — didn’t understand how to use Verify until the second question.)
5. Set an Outlook reminder to yourself to reset the test on the 6^th day. The maximum number of days a test can run is 6 days, but you can reset it to run seamlessly. This makes no sense to me, but that’s how it works.
6. Allow for more participants than you actually need. Several of your participants will figure out that they can take the test again and again — especially if a gratuity is involved.
7. There is no way to automatically turn off the test once you hit your quota, so unfortunately you have to watch the test results like a hawk.

Verify’s customer support treated me very well. I contacted Verify’s support team on more than one occasion. They were always quick to respond (usually within 24 hours), and plainly answered my questions.

My Overall Grade for Verify: B-

If you’re short on time and budget to conduct in-person focus groups or moderated usability studies, I found Verify to be a satisfactory solution to use in a pinch. There are definitely online research tools that are more powerful, more customizable, and make analysis infinitely easier than Verify.

So… Which Test Won?

I am pleased to report that all of our new ideas beat the pants off the control (the current landing page). We were able to obtain clear insights into why some concepts were preferred more than others, and what exactly people don’t like about the current landing page.

Our next steps are to present the findings to the leadership team and propose a small A/B test to see if customer behavior bears out.

Ah ha, the true test!

Related Articles

What to Do When A/B Testing No Longer Yields Results
28 User Research Methods
When to Use Which UX Research Method