How to get started with A/B testing & Personalization with Sean Bestor

A/B testing & Personalization are indispensable tools for today's marketer. That's a fact. But how are the best marketing experts really using it? How about the blogs you read and tools you use every day… let's say Sumo (ex-Sumome) for example? Welp, wonder no more. We asked Sean Bestor (@sbestor15 on Twitter), Head of Content at Sumo his thoughts on A/B testing & Personalization and what they do with them.

The interview

Question: How long have you been working at Sumo and what does your job consist of?

Sean: Over a year now, and it’s a mix of different marketing disciplines. We keep things lean, so in any given week I’m writing a guide, writing emails, helping with webinars, conducting A/B tests, writing for the product or support teams….it goes on and on.    

Q: Could you provide some context on how Sumo came to be? I understand that it got created by the AppSumo team? And why did you change your name?

Sean: We created a lot of tools to grow AppSumo back in the day. One day we said, “Hey, this stuff worked for us, maybe other people want these tools, too.” So we created Sumo and gave away all those automatic site-growing tools for free (we STILL give them away for free). Noah wrote a great article on why we changed our name to Sumo, but one other reason was to show how serious we are when it comes to investing in our users. We’re doing big things in 2017, and we wanted our name to reflect that. If we spend $1.5 million on a domain name, imagine the big stuff we’re doing for the product…    

Q: You guys basically make tools for people to grow their websites. What kind of tools exactly and how advanced can they be?

Sean: Think of Sumo as a one-stop shop for automating your site growth. Right now, you’re probably subscribed to a ton of marketing tools. Not only are you paying way too much for those tools, but it’s a pain in the ass to log into every single tool, pull all the data and make sense of it all — much less connect them in any sort of way. Sumo takes all the essential tools you need to grow your site and puts them all under one roof. It’s like installing a butler on your site and saying, “Chop, chop! Make my site grow.” We have tools that help you:

• Get more customers: Email is the most proven way to consistently turn visitors into customers. When it comes to capturing emails, there’s no better tool on the internet (especially since the majority of our features are free).
• Get more traffic: Want your hard work to be…you know…seen by people? We’ve created tools that can instantly increase your traffic by 20% (and did we mention we’re SEO and mobile friendly?).
• Track your success: Publishing and praying aren’t the way to operate. Our tools show you exactly where your visitors click, how engaged they are with your pages and if they really read everything.

And they’re as easy or advanced as you want to make them. We have some users that take 5 minutes to set up a popup, they let it run and they get thousands of new emails. THEN, we have users that create crazy rules to only show certain popups to people coming from a specific Instagram post. It’s like any other skill — the more time you put in, the better your results.

 

At Kameleoon we sell A/B testing & web personalization solutions, both these practices are an integral part of Sumo’s tools.
 

Q: What is your definition of web personalization?

Sean: People have complex answers, but really it comes down to this — delivering the best experience to the right people. >>> Great experience + wrong audience = Wasted content >>> Bad experience + right audience = Waving bye-bye to that audience forever Neither are preferred. And there’s no real, definitive “great experience” I can point you to, because it depends completely on the audience. That’s why listening to them is paramount — you can’t have a great experience without finding your perfect audience first.

Q: How do you guys use it to improve your visitors’ experience?

Sean: Here’s an example we use for our guides. We use Heat Maps to see if people are clicking on our content upgrade buttons (if they aren’t, we change button text or design). We use Content Analytics to see how far down the page readers get (if it’s not far, we adjust the length of our articles…and we also see the best place to put our content upgrade button). Then we use List Builder to create a personalized popup (testing headlines, images, button copy, etc) that only appears to select, engaged readers. And that’s just for our guides :)    

Q: Could you share an example of a personalized experience that had a considerable impact?

Sean: Recently we changed the design of the button we use because users said they had trouble finding our content upgrade opportunities. So we turned our text box into a big button and saw a massive increase in clicks (which resulted in more opt-ins).    

Q: About A/B testing, how do you use it to grow Sumo? How often? How many tests do you have running right now?

Sean: We A/B test all of the email capture apps we use on every single piece of content. That means that on any given time we have 1-2 A/B tests running on each piece of content to find out the effectiveness of different elements to our email subscriber conversion rate. Since we publish on the blog weekly, we’ll set up a new A/B test every week. We have dozens of A/B tests running right now.    

Q: Could you share details on an A/B test that worked really well?

Sean: With one A/B test, we tested a small Click Trigger pop-up against an extra-large pop-up to see which would perform better. The results surprised even us! The extra-large pop-up performed 227% better than the small! The funny thing is that it didn't work so much better because the small one was not noticeable. It's a Click Trigger, so our visitors had to click a link to bring it up.    

Q: What does your optimization process look like? Is it something clearly defined or do you have general guidelines?

Sean: Each month we go through all our tests and look over the results. If we have some clear winners, we work to implement that throughout the whole site. If we have losses, we try new stuff to see if we can raise the results. It’s not a hard process, and it doesn’t have to be.    

Q: What are the next steps for Sumo?

Sean: We’re going to change the way you think about personalization — not just for your visitors, either.    

Q: Any parting words for marketers out there getting started with user experience optimization?

Sean: Spend the time upfront to identify where your areas to improve are greatest and don’t expect big wins right away. It’s sexy to talk about big results, but they’re few and far between. Small wins are like rungs in a ladder. The more you have, the closer you get to that big win.

 

There you have it folks! Hope you enjoyed as much as we did ;)

If you want great tools to grow your website check out our pals at Sumo.com.