How We Drive Massive Growth by Running Calculated, High-Risk Tests

The Wishpond growth team is made up of a bunch of great and (frankly) pretty ballsy people.

It’s Carlo, Kevin and I (Wishpond’s content team), Matt (head of optimization), Nick (CMO), Ali (Wishpond founder and CEO) and then Marya (our overworked marketing/UX designer).

At the beginning of each month we have a brainstorming session dedicated to growing Wishpond’s new revenue.

We sit down in the boardroom and come up with ideas, write them on the whiteboard and then run an ICE evaluation of those ideas.

Ali runs it and we all come prepared. Even so, with several coffee and stretch breaks, it takes us more than two hours from beginning to end – but it establishes what a lot of those people do with our days in the coming month.

This is all pretty standard…

The difference between this meeting and those I’ve attended at other companies is that we don’t mess around with low-impact tests. We change prices. We overhaul design. We change the flow of our website and the flow of our conversion funnel. And we do it every single month.

Unless you know what you’re doing, this kind of growth marketing can bite you in the ass.

This guide will break down the exact process we use to run calculated, high-risk, high-impact conversion tests.

Let’s get into it!

Starting the Process – Where we Get our Ideas

There are three primary strategies for brainstorming good growth strategies. I’ll rank them in the order of impact…

1. We Steal:

I’ll give you an example…

About a month ago I was doing my ordinary visit to the websites of companies I love (as well as a few competitors). I do it at least once a week, as well as subscribe to their newsletters, to keep up on what they’re developing and what seems to work for them.

Before I dive into this, you need to know that it’s not enough to create a testing hypothesis based solely on “so-and-so did it so it’s worth trying.” You need to come up with a legitimate justification beyond that.

But it can be the inspiration; the reason you explore the idea.

So I notice that Klientboost has moved their signup form and flow from a new page of their website into a modal window (an overlay which pops up when you click “sign up” or something similar).

Given that we’ve had a lot of success with modals/click popups on our blog, this was more than enough for us to start our own tests following their lead.

The top 5 companies I keep an eye on for “inspiration:”

2. We Follow Instructions:

There are a half dozen fantastic growth-minded resources from which we take direction. In general, if Rand Fishkin says something and shows his work, the strategy is worth looking into.

For us, it’s all about the people who are showing their work. Our growth team is, predominantly, beyond “best practices.” Yes we keep them in mind when designing a test, creating content, or optimizing our website’s flow. But we (and you) shouldn’t blindly trust anything anymore.

Your sources need to show their work.

The top 5 people or sources we trust the most (with examples of why):

Rand Fishkin, of Moz: Minimum Viable SEO
Brian Dean, of BackLinko: The Content Relaunch Walkthrough
ConversionXL, particularly Ben Labay’s original research pieces
Neil Patel, of Quicksprout: Using 404 Pages for SEO
Drift Podcast: Literally all of them.

An actionable strategy for following instructions is simply to add your top blogs/writers to Feedly (or another content collator site) and keep it bookmarked.

3. We are Actually Creative:

This is third and final for a reason.

I have to be honest. Very few of our growth strategies come from scratch.

What does happen is that things we see lead to creative ideas. Tests lead to tests.

And that’s one of the biggest things for you to take away from this post (I’ll repeat it): Tests lead to tests.

But that requires you to keep a good record of your tests. Every loss, tie or win is a lesson you can use to create better tests down the line.

Trello or Asana are great for this, as they allow you to quickly and easily see the status of each of your tests, assign ownership of those tests to different team members at appropriate times, and note any wins or losses.

Both platforms (for this purpose) work on a column system where you can see the progression of your tests and make notes on each.

We also have a column after “Test Won,” where we place tests, with full notes, which we think can be implemented again in a different place or way.

How We Determine Priorities with The I.C.E. Process

The I.C.E process works simply:

Take a look at your ideas and then judge them on three parameters:

Impact Expected: How influential will this test be on our ability to meet our 10% growth goal?
Confidence: How confident are we that the test will drive that level of impact?
Ease: How easy is it for us to implement this test? Will it be a simple copy change in the backend, or will we need to get a developer on board?

Total their score, and if they’re high-impact, you’re confident of that impact, and they’re not impossible, the ones most likely to effect your bottom line will win.

After we’ve brainstormed and ICE’d our ideas, we do a bit of Excel magic and determine which 10 or 20 ideas we’ll be dedicating our next month toward as a team.

Here’s what the growth ideas sheet looked like for April, 2017:

We move the ideas which win (10-20, depending on Ease) to Trello or Asana.

Bottom-of-Funnel Testing: High Impact, High Confidence

Each and every time, the ideas based around our Pricing, Signup, and Billing pages are the ones which we prioritize.

That’s why, in the past couple months, we’ve run at least a dozen (big and small) tests on those pages.

Your bottom-of-funnel pages are the ones which affect your bottom line the most. A 5% improvement in the number of people who complete your billing form is, by definition, a 5% increase in your business’ new revenue.

Unless you have a million or so site visitors each month, don’t sweat the small stuff. It’ll take too long to reach statistical significance and won’t affect your bottom line enough to matter.

Instead, make serious changes – changes which go beyond “let’s change our CTA button from red to orange.”

You can optimize your landing pages, product pages and homepage for a year and not see a 5% increase in revenue.

And that’s because a 50% increase in clickthroughs on your homepage to product page (for instance) may not actually result in a sizable effect on your bottom line.

The Danger of Running High-Impact Tests

High-Impact tests are, by definition, high impact.

If they are good; if they line up with what your prospective customers are thinking; if they make the process easier or make it harder to say no, they can increase your revenue by 5, 10, even 20% overnight.

If they don’t, of course, they can drop your revenue by those same numbers.

How to protect yourself from the danger of high-risk testing:

Split the split: Most A/B tests show your variation to 50% of your traffic to any given page. If something is seriously scary, show it to a quarter of your traffic instead. It might take a bit longer for your test to reach statistical significance, but you won’t get burned as badly if the test burns.
Keep a very close eye on the results: High-risk tests aren’t ones you can set and forget. You need to be checking every hour or two to see if you need to pull the plug.
Run intelligent tests: The last thing I want you to take away from this article is that I’m recommending you to cover your entire homepage with “BUY!” buttons and walk away. Be smart about this. Not every high-impact test is going to work, and not every high-impact test is one you want to run.

For instance, our growth team has discussed adding countdown timers to our pricing page, giving visitors an incentive to buy before a discount expires. Even if that won, we decided the damage to our brand reputation would negatively impact us more than we’d profit.

What Bottom-of-Funnel Pages to Test Each Month:

Pricing Page
Plans (if different from Pricing)
Signup Page
Checkout (if E-Commerce)
Billing Page

High-Impact Test Ideas:

Test reordering your conversion funnel: Send your blog traffic to your pricing page. Send your webinar leads an immediate sales call instead of nurturing them. Best practices might not actually be best for you, and testing allows you to see what’s what instead of assuming what’s probable.
Test a complete visual overhaul or redesign: Change the length of your page. Add a background image. Remove text. Add text. Go big or go home.
Test your tone: Add humor. Add cartoons and mascots. Make the whole thing cartoonish. Identify your target market and test changes to appeal more to that market’s sensibilities.
Test how people can sign up: Add the ability to sign up with Google, Facebook, Twitter, Linkedin, etc.
Test how people can pay: Add more payment options (this is particularly important if you’re venturing into the international market. A significant number of potential customers, for some businesses, may not have the most popular North American credit cards).
Test how you give access to your tool: Give people access to your software before entering their billing information. Only prompt a free trial or payment when they try to launch.

An Example…

Control on the left, variation on the right:

This test increased our pricing page conversion by 54.3%. Check out our breakdown of this test in my colleague’s article “How We Redesigned Our Pricing Page to Increase Revenue 54.3%.”

What You Need to Make This Work: Tools and Expertise

Let’s quickly break down the things you need in order for this to work.

1. Time

More than anything else, testing takes time. It takes time to build a test which has a chance of winning, and it takes time for that test to reach statistical significance.

But you can’t skimp on that second part. I know it’s tempting, but unless you’re extremely confident of a test’s chances, don’t announce it a winner until at least 90% significance. Your testing tool, below, will help you with this. Alternatively, check out Get Driven’s Test Calculator.

You can, however, skimp on the first one (taking time to build). Bear with me…

Your A/B tests can be a proof of concept. They don’t, necessarily, have to be your final, 1000%-ready editions. This is particularly relevant when we’re talking about design tests. A lot of businesses (ours included) prefer to move fast and intelligently than slowly and extremely carefully.

Create something which looks good, and then test it. If it wins, you can (usually) be confident that a more complete design or more complete implementation of that concept will win more dramatically.

2. A testing tool which makes it possible

For us, this is Optimizely. Though we’re starting to play around with Google Optimizer as well. If anybody’s used it and wants to share their experience in the comment section, we’d appreciate it!

Optimizely allows you to, without a need for a developer, make simple design and copy changes on a page variation of your website.

We reached out to Optimizely VP of Product Marketing, Brigitte Donner, for her take on where A/B testing fits into business growth. Here’s what she had to say…

There’s a clear link between testing and fostering a culture of experimentation and the creation of actionable insights that help to drive business growth. We are seeing customers of all sizes and industries profit from running multiple experiments across all of their digital customer touchpoints. We find that clients see the best results when they test throughout their entire marketing funnel. It’s not enough to just focus on acquiring new customers, but the total experience of the customer journey that is paramount to success.

We recently conducted a study with Nucleus Research and discovered that our customers have been able to quantify the direct impact experimentation has had on revenue growth. One retail company, for example, experienced a $200,000 boost in revenue due to one experiment it ran in 2016. This is an exciting statistic to see as with the right expertise and investment in place, the value of a robust experimentation program can add back significantly to the revenue creation potential of digital platforms.

Brigitte Donner, VP Product Marketing at Optimizely

3. Someone to design and implement the tests

For us, this is Matt and our marketing developer, Jacky. I’m not saying that Carlo, Kevin, Nick and I don’t run our own tests. But they’re involved (to a greater or lesser extent) in all of the big tests we run.

It’s because testing is more complicated than I wish it was. Optimizely (or whatever A/B testing software you use) will allow you to do a lot of simple changes, and those can be very powerful.

I’m talking about CTA button text, page copy, even the more straightforward design stuff (like color).

The challenge, though, comes when you want to stop sending people from your homepage to your product pages, but want to instead send them to an “Explore” page covering all your products in one place. So you want to remove the Products dropdown in your Nav Bar and instead link to your new Explore page beneath your homepage’s headline.

That’s a bit more complicated, but also more high-impact.

First and foremost you need to run every significant test by a designer with a UX background before you launch. You might think that the color you’ve chosen works and will improve conversion rates (and it might), but even so, run it by someone who went to school for this stuff.

Secondly, if your tests are a bit more complicated, you might need a developer to actually set them up.

My recommendation for this is to have one of your business’ developers be a part-time “marketing developer” who has the responsibility of both product and platform development but also anything your marketing team needs.

Trello/Asana are great for this stuff. They allow you to assign a job to your designer or UX guy after you do your concept mockup, and then to your developer to put it into action.

Otherwise, you find yourself excited to launch a test in which you change your signup process from a new window to a click modal and are hit by “That’s not a priority. Maybe next month.” and your growth could stall as a result.

Make sense?

Wrapping it Up

Hopefully this guide has given you a better idea of how we structure our own growth strategy. A few things to keep in mind:

Steal growth ideas when your creativity runs out.
Identify the top thought-leaders in your industry and keep abreast of what they’re recommending.
Test the big stuff. Don’t beat around the bush with your tests. Go big or suffer the consequences. Multivariate testing (testing of multiple elements on your page) is, in my book, fine for about 75% of smaller businesses. Unless you multivariate test you’re not going to get anywhere fast.
Be sure your tests reach statistical significance, unless you’re super confident and are willing to risk a false positive.

And good luck! Let me know if you have any questions about our strategy or what you should test in the comment section!

Related Reading:

james-scherer-2