How Wistia used A/B testing & Personalization for conversion optimization

When it comes to cool, fast-growing startups, Wistia, video platform for businesses, is definitely in the conversation. They've been ranked on 2017 Top Company Cultures List Presented by Entrepreneur and CultureIQ and serve more than 300,000 around the world! And it sure makes sense, considering video is all the rage today in Marketing. The general consensus is that video will dominate almost every online channel. How does a startup like this grow its website? Do they use A/B testing? Personalization? How do they think about conversion optimization? Wonder no more. We had the chance to interview Andrew Capland, director of growth at Wistia. And let me tell you, he didn't hold back.

The interview

Question: How long have you been working at Wistia and what does your job consist of?

Andrew: I’m the Director of Growth at Wistia, and have been here for two years. Growth at Wistia involves the whole funnel; from new account creation to new user activation, and increasing our customer count and corresponding monthly revenue. So, depending on what our specific goals are at the moment, we might work on projects on the Wistia website, in the Wistia application, email, etc. We spend a lot of time analyzing funnel data, collecting user feedback, and working on projects/ experiments to increase conversion.    

Q: Wistia is a video platform with analytics and video marketing features. What made your success? What’s next for Wistia?

Andrew: The company has always been focused on providing tons of value to our users. I think that’s been a main driver of our long-term success. You find that in every step of the Wistia experience. From our marketing, where we’ve created tons of really helpful content with the goal of helping people make better videos. You’ll find it in our product, where we put a premium on user experience and making things clear & easy. You’ll find it in our customer support, where we put a huge emphasis on making sure every interaction (from emails to phone calls, to our user conference WistiaFest) is focused on helping & delighting our customers. That really comes from the top down and has been a main growth driver for the company. In terms of what’s next, I don’t think that’s going to change. But, we have some great new features and integrations in the works coming soon!    

Q: Video is getting incredible traction across all channels. How do you see it evolve in the next couple of years?

Andrew: Creating video used to be really hard and was something only professional videographers could do. Now, it’s becoming easier to produce high-quality video, so we see more and more video being used in the future. In the future, we think every business will be using video to increase their sales & communicate more effectively.    

Q: How does video fit in the quest to optimize website visitors’ experience? Where can people start?

Andrew: Video is a really powerful communication tool. So, it’s only natural that it can help improve your visitor experience. We use a ton of video throughout our user journey; in our marketing materials, to drive product adoption, and to communicate important information to our users. We’ve spent a lot of time figuring out which videos would be most effective. For people just starting out, I’d start by identifying an experience that you think is confusing for your users, and then design a video to help reduce that confusion.    

Q: What are your go-to ways to identify optimization opportunities on your website and blog?

Andrew: We’ve adopted a modified ICE score methodology, which was made popular by Sean Ellis. It always starts with a problem or opportunity. Like, “I think we can get more new account signups from this page, here’s why…” That “why” can be user feedback from surveys, it can be low conversion rates, or it can be applying learnings from other areas in your user journey. Once you’ve defined the opportunity, try to quantify it. “If this project goes perfectly, what happens?” And, put a real number associated with it. That way you can easily prioritize a few different opportunities against each other. Finally, we work with our development team to estimate how much time we think the specific project will take. If it’s a quick/easy project that has a big impact - we prioritize those first. If it’s a long/challenging project or has a smaller impact - it’s easy to keep on your backlog for a rainy day.    

Q: Are you using A/B testing to optimize them? How often do you run experiments?

Andrew: Yes, we do a lot of a/b testing on our website and in our product. Things fluctuate depending on our goals and how quickly we can iterate. But, we typically run a few tests a month.    

Q: What do you look for in an A/B testing tool?

Andrew: For us, the most important thing is having clean data. Making sure it’s easy to set up and target specific audiences to get that clean data is the number one thing we look for.    

Q: Could you share details on a recent successful A/B testing you conducted?

Andrew: We just spent a lot of time optimizing our pricing page. We ran a series of a/b tests to help us determine the proper style, copy, and content to include on the page. One specific test that was successful was adjusting our FAQ copy on the page. We ran a survey on the page and learned many users did not want to create (or purchase) an account because they had lingering questions they couldn’t find the answers too. We got a list of those questions from the survey, then designed a test adding those questions/answers to the page. We found out that by adjusting our FAQ copy we were able to increase our signups from the page by 8%. Over the years, we’ve found accounts created from our pricing page tend to be worth a lot more than other pages, so this was a major win for us!    

Q: What was your biggest mistake with A/B testing?

Andrew: Poor test setup. In the past, we’ve targeted incorrectly or changed things too soon, and those mistakes cost us valuable time that we could have spent in more productive ways.    

Q: What are your thoughts on Web Personalization?

Andrew: Using information about your users to customize their experience is a win-win. Your users win because they get information that’s more likely to be helpful to help. And, you win because those will be better users that will turn into better customers.    

Q: Do you use personalization to grow Wistia? If yes could you provide an example? If no, why not?

Andrew: We do some personalization during our new user experience. We collect certain information on our signup page, then customize the user experience and highlight tools/features we believe will help our users be successful.    

Q: Any parting words for marketers out there getting started with user experience optimization?

Andrew: Start small and focus on learning. If you focus on learning both how to run proper tests and also what resonates with your audience, you’ll be setting your company and your career up for success. Be patient!

Conclusion

Hope you enjoyed it as much as I did! What's your biggest takeaway from Andrew's answers? If you want to know more about Andrew, check out his website. Wistia also recently released an awesome chrome plugin to record, edit and share videos super-fast, called Soapbox.