Kameleoon’s take on performance and rankings
For over a decade, Kameleoon has been leading the experimentation space by prioritizing performance. Our commitment to speed and seamless customer experiences earned us consistent top rankings. However, a sudden change to the ThirdPartyWeb rankings, an OpenSource performance benchmark, caused a surprising drop in our ranking. This was surprising, as we had made no changes to our platform that could explain such a drastic shift.
After investigating, we discovered that the methodology behind this OpenSource ranking had been altered and introduced by an experimentation vendor which historically ranked poorly. The specific change can be publicly viewed here.
More importantly, this vendor has reworked the execution of its scripts giving a completely misleading view of its impact on both performance and the user experience.
Their script has now been split into multiple parts : an initial, almost-empty script loads quickly on the page, followed by five or more additional scripts that load all necessary targeting and experiment configuration data.
This process results in a significantly longer load time, causing a pronounced flicker effect which is not monitored in the ranking.
Understanding the full picture of performance
Although benchmarks provide useful insights into CPU impact from third-party JavaScript, it doesn’t capture the full picture, especially for experimentation platforms.
Real performance depends on balancing speed, accuracy, and delivering a seamless user experience. This is where Kameleoon strikes the ideal balance: Our experiment snippet is designed to optimize both performance and user experience.
First, Kameleoon loads one, and only one script. No additional script is being loaded afterwards.
Second, Kameleoon’s script embeds a unique anti-flickering engine, which ensures A/B tests run smoothly without delays in displaying test variations and directly contributes to a better user experience and more reliable test results.
Temporarily disabling our anti-flickering engine improves performance significantly. However, this improvement comes at the expense of the quality and accuracy of the test results which is not an acceptable trade-off to our opinion. To learn more about why Kameleoon’s anti-flickering technology provides the best user experience, see our blog article.
Additionally, our Deferred Experiment Loading feature offers flexibility, allowing customers to defer certain experiment configurations—like pop-ups or elements below the fold—that aren’t needed immediately upon page load. This helps optimize performance without sacrificing user experience.
Enterprise-Level Performance
For enterprise teams running large-scale experiments, performance is crucial. Kameleoon’s snippet is built with cutting-edge technology, making it asynchronous and fully cached, ensuring page loading is never blocked, even during CDN incidents (99.99% uptime). Our compression techniques (combining Typescript and Brotli) deliver a lightweight 28.4KB script for minimal resource load.
Additionally, features like Beacon calls (204) reduce network requests, and our Deferred Experiment Loading allows selective experiment inclusion for faster initial load times. All of this guarantees seamless performance without compromising functionality, making Kameleoon the ideal platform for enterprises focused on speed and user experience.
Looking ahead
Our mission at Kameleoon is to deliver an experimentation platform that balances speed, accuracy, and reliable results. Reach out to our team to explore how Kameleoon has the fastest script on the market to help you achieve your business goals.
