How SDKs enable experimentation everywhere

As the benefits of A/B testing and experimentation grow, more and more brands are beginning to optimize not only their frontend UX but also the backend functionality of their digital products.

However, running A/B tests on the platform itself, for example when testing features on a mobile app, often requires a lot of custom development. Long development cycles, in turn, mean fewer tests that take too long to go live. As a result, your testing velocity suffers. And so do your insights and returns.

SDKs fast-track development. SDKs from full-stack A/B testing solutions like Kameleoon give your developers all the tools and the infrastructure they need to quickly create and deploy test builds to a host of non-website channels.

SDKs help you take your experimentation outside of the web browsers—to your desktop, mobile, and web apps (even the single page ones). They power deep product-side experiments by bringing different touchpoints along different channels straight into your experimentation program. SDKs let you experiment everywhere in the real sense—within the server, mobile app and browser.

1 What are SDKs

Software development kits (SDKs)—also known as devkits—are packages that help make two solutions work together seamlessly.

An example:

Suppose your solution has an iPhone and Android app and you want to test a new feature or an app experience on them.

Now:

• Your Android app would probably run on Java or Kotlin (which has replaced Java as the official language for Android app development). Your app could also use C++, C#, or Python.
• On the other hand, your iOS app's programming language could be Swift, Objective-C, or C#.
• Your apps could also use frameworks like React Native or Node.js.

In order to use an A/B testing tool to run your experiments on your mobile apps, you'd need it to work well with your applications' core technologies—their programming language(s) or framework(s).

SDKs are like middleware that facilitate this connection.

SDKs work as a kind of middleware between your experimentation solution and your test's production environment. In this case, your apps. They're like a bridge connecting your experimentation software to your production channel where your experiments will actually run.

SDKs provide your developers all the components — APIs, libraries, editors, compilers, debuggers, sample code, and documentation — they need to quickly take your experiments to all your channels and platforms.

Developer-friendly solutions try to offer up-to-date and actively maintained and developed SDKs to support all the popular technologies (programming languages, frameworks, and platforms) their customers use.

2 How SDKs work (and why they enable experimentation everywhere)

Once you install and initiate your experimentation solution's SDK for a target channel, setting up experiments on it is very easy.

If you wanted to set up an iOS app experiment using a solution such as Kameleoon, you would get all the code and methods/function you need packed right inside our iOS SDK.

All you'll need to do is code the experience(s) you want to test — or the test build(s). And set up and map the necessary instances and properties between your app and Kameleoon. The code to make both work together is already shipped with the SDK.

Here's how it would roughly work like (post the SDK installation and initiation steps):

1) The Kameleoon dashboard lets you create a new test.

2) Kameleoon's iOS SDK gives you the code to trigger your experiment (available via the triggerExperiment() method).

3) To manage your variation code paths/experiences, you'd need to pick the variation ids via Kameleoon's intuitive dashboard and insert them into your build code using simple if...else or switch mechanisms.

The documentation you get with the devkit comes with sample code and best practices to assist with these steps.

4) With the triggering and variation code paths set, you can use the iOS SDK's trackConversion() method to track your conversions.

These get logged into Kameleoon automatically. Kameleoon also has bridges with Segment.io, which can be fully automated thanks to our Segment Destination Bridge

5) Finally, all the reporting happens inside Kameleoon's dashboard just as it would in normal website-based or front-end experiments.

In most cases, your experimentation solution's dashboard should let you easily launch, pause, and stop your experiments, no matter what channels they run on.

SDKs do most of the heavy lifting for running your experiments over different channels. They give you seamless integration between your experimentation solution and your target environment's technologies so that your testing campaigns get orchestrated well.

SDKs vs. APIs: Because of the level of support they offer to make two solutions work together, SDKs differ greatly from APIs (that mainly only facilitate the relaying of data between two solutions). SDKs also do not impact latency as they do not use blocking calls, which APIs do.

3 Not all SDKs are created equal

While all SDKs come with more or less the same components, not all SDKs are created equal. A few considerations:

• Performance: Just like any bloated codebase, even SDKs can cause performance issues. Additionally, they can cause latency if they aren't optimized for memory and network usage. Before choosing an experimentation software, ensure that it has a low footprint. Even better if it comes with a zero-latency policy.
• Documentation: Documentation isn't just an essential component of an SDK, but its quality determines how easy the SDK will actually make your integration and development work. Observing the sample code provided in an SDK's documentation and the handling of exceptions can save you a lot of headache.
• Maintenance and regular updates: Unless SDKs are compatible with the recent versions of the platform or technologies your solution runs on — if not with the latest one always — they're no good. It's important to review your potential experimentation solution's SDK changelogs to see how invested they're in maintaining them. Regular bug fixes and enhancements, too, are a hallmark of good SDKs.

4 Top SDKs for A/B testing and experimentation

Broadly, you have three types of SDKs when it comes to experimentation: Mobile SDKs, client-side SDKs, and server-side SDKs. You could club mobile and client-side SDKs as one category.

Of these, client-side SDKs work directly and are embedded within the applications your users use, including your web, mobile, and desktop applications. This is why mobile SDKs also count toward client-side SDKs. Typical examples of client-side experimentation SDKs are JavaScript Library, React, and Angular.

Server-side SDKs, on the other hand, live on your tech stack (on your servers or the services you use on the server-side). They're embedded into your back-end tech infrastructure. Some examples of server-side experimentation SDKs are .NET, Node.js, and Java.

These are a few popular mobile, client-side, and server-side SDKs that an experimentation tool should support:

• Android (Kotlin/Java)
• iOS (Swift)
• Flutter
• Java
• C#
• PHP
• NodeJS
• Ruby
• Python
• Go
• JavaScript / TypeScript (ReactJS, Angular, Vue)

5 SDKs need regular updates and maintenance

Evaluating SDKs when choosing an A/B testing or experimentation solution

Product and dev teams work below the front end, unlike marketers and salespeople who use experimentation solutions to test surface-level elements like different layouts, copy, offers, etc. They want to know if they can replace the current search algorithm powering the app or use a different functionality implementation or introduce a feature they just hypothesized.

SDKs: the key selection criteria for technical stakeholders

That's why technical stakeholders look for more than shiny visual editors or code editors that allow easy HTML/CSS injection. For them, an experimentation solution's true technical prowess lies in breaking the browser's barriers and bringing experimental experiences everywhere.

SDKs = Quicker test deployment across different channels

Product-led businesses that run several experiments each month look for developer-friendly experimentation solutions that offer SDKs to maintain a good testing velocity while also controlling costs. SDKs support such teams in launching and deploying multi-channel and multi-environment experiments seamlessly by offering all the needed groundwork.