After almost 3 years working as a CRO developer, progressing from an assistant to a senior, the time has finally arrived for me to take on a new challenge as a Front-End developer. Over this time I’ve learnt a lot about building A/B experiences for use with different testing/personalisation platforms, such as VWO, Google Optimize and Monetate/Kibo Personalization. I thought I’d put together a little post to share some of my top tips, specific to building such experiences.
If you’ve never had any experience with building A/B experiences before, you may wonder what the differences are when compared to the duties of other Front-End developers. Basically, development of these experiences is usually limited to creating JavaScript (JS) and CSS to serve on top of the existing website, as a result your problem solving and creativity as a developer is sometimes really put to the test.
When I first started, we were using one JS file and one CSS file which were stored on Microsoft’s OneDrive cloud storage for backup. Luckily, thanks to some great members of the team, we evolved from that and started using a tailored solution using Git, Webpack, SASS, Babel and other technologies. Both approaches worked perfectly fine, but the development process was drastically improved in many ways by making the switch.
I never ended up fully dabbling into server-side testing, so most of my experience and tips relate solely to client-side testing.
Tip: Poll/Wait for pretty much everything.
As the experiences we create are getting served on top of the existing site via a script from the testing/personalisation platform of choice, this means that our JS and CSS will be executed as soon as it has loaded. We have no control over what will happen before, or what will happen after our scripts execute. This is why it’s super important that we poll/wait for anything external that we need before we try and utilise it.
For example, say that we are building an A/B test where we will move a third-party reviews widget to the top of a product details page rather than the bottom. We would use a poller function, like these functions from David Walsh for example, to wait for the reviews widget to exist on the page before we try to move it. Another perfect use case for polling is when using third-party scripts that you expect to be on the site. For example, if we ever need to use jQuery from a client’s site, we can use a poller to wait for jQuery to load before trying to use it.
By default all of the A/B experience builds that I create first and foremost poll for the “body” element. Believe it or not, in the past I have had experiences causing errors because the script loaded before the “body” element was ready.
There are two main ways that polling can be implemented:
- Poll for each thing individually as and when it’s needed
- Poll for everything needed before executing the rest of the script
I tend to do the latter, the issue with polling for each thing individually is that timing issues occur, sometimes things won’t load in the order expected and it can cause the rest of the script to get confused. Using the second method of polling for everything before executing the rest of the script, allows me to keep all of my polling functionality in one place, and it doesn’t cause any timing issues throughout the script.
I would say that polling is the number one tip, without it CRO development would be almost impossible.
Tip: Get to grips with the Mutation Observer API
The Mutation Observer API is one of the most useful tools that I use when developing A/B experiences, so knowing how to utilise it is a must! As CRO developers tend to have no access to hook into any of the client’s functionality, we can’t know when to fire events based on a user's input. Mutation Observers allow us to watch the DOM tree for changes, and fire a callback function every time a change has been made.
An example of where a Mutation Observer would be useful would be on a product listing page (PLP). Say the A/B test was making changes to all of the products within the PLP, If we only fired our changes on page load, as soon as a user changes the filters, changes the sorting method or loads more results the changes would be wiped (depending on the site’s functionality). With a Mutation Observer, we can watch for every time the listings change and re-apply our changes to each listing within the PLP.
Above is just one example of how the Mutation Observer API can be super useful for CRO development, but I encourage you to take a further look into the docs and familiarise yourself with all of its capabilities. I have also previously written a blog post all about Mutation Observers and how they help during CRO development.
Tip: Take QA Seriously
Carrying out QA on all of our experiences is a must, but there are a few things we need to do to make it worthwhile.
It goes without saying that we should do small checks on our own work before we pass it over to be properly QA’d, but the idea of the QA is to get someone with fresh eyes to take a deeper look at the experience across multiple devices and browsers. As developers it is our responsibility to set the QA environment up and inform the person carrying out the QA of what needs to be checked.
It is super important to carry out the QA in a live environment, and not to carry out the QA on the testing/personalisation platform’s “preview mode”. From experience with all testing platforms I’ve used, the “preview mode” loads significantly slower than when the experience is actually live, with some platforms even ignoring the set targeting rules when in “preview mode”. This makes the experience that is tested completely different from the one that will actually go live. Most issues that arise from this loading discrepancy can actually be nullified by implementing the first tip, polling for everything, but it’s still important to QA in the live environment.
You can QA safely in a live environment in multiple ways. We usually opt to set an experience live to 100% and use targeting to only show that experience if there is a specific QA query parameter or cookie present. Doing this allows us to have the benefit of QAing in a live environment whilst still hiding the experience from any other site visitors.
The other thing that I recommend is to document your QA process. We have a document with all tasks that we check per device/browser and we confirm that all of these checks have passed for all devices/browsers. On top of this, we take multiple screenshots for each device/browser. Documenting the process really provides a safety net if an experience breaks after a few weeks of being live. If we can prove that the experience was all good when we launched it, it allows us to show that the client has made a change to their site that is interfering with our pre-existing A/B experience. Of course our experience will still need fixing, but it allows us a bit of time whilst we sort it out.
Tip: Dig around the site’s existing code
Depending on the experience that I have to build, I sometimes spend some time before I even start the build digging around the site’s existing code to see if there are any exposed methods or functions that I can use. Clients are usually quite reluctant to help you when it comes to building an experience, so for the most part you’re on your own.
I only search for things relevant to functionality needed for the A/B experience I’m building, obviously. I’ll use Boots website for an example, I can see here that they have zoom in/zoom out buttons on the product image (outlined in red)…
Let's say that for the A/B experience we’re creating we want to add 2 buttons elsewhere on the page to allow a user to zoom in/zoom out of the product image. Well, from a quick look around in Chrome Developer Tools (other developer tools are available), I can see that the original zoom functionality is exposed…
Knowing this, we can easily add buttons elsewhere which will utilise this functionality and saves us having to implement our own.
This is a really basic example, but doing this can be really powerful and save lots of time. Suggestions to get started at finding existing functionality would be to look at the events attached to an existing button, look at network requests for any APIs or even try typing in “window.” to the Dev Tools console to see which autocomplete suggestions appear.
Tip: Do your research on a page type before beginning the build
My last tip is to do your research on your page type if the A/B experience will be targeting multiple pages of the same type. For example if the experience is targeting all product detail pages (PDP) on a site.
Doing this step is really important, as it allows you to check for discrepancies that may occur across different pages of the same type. The assumption is that all pages of the same type will have the same content or classnames/IDs as each other, but so many times in my experience that hasn’t been the case. This causes issues when some JS/CSS you create is specific to an ID that doesn’t exist on every page, as your changes will then not get applied or might even cause an error.
Hopefully any issues caused by this would be picked up during the QA process, but catching it out before beginning the build can save time in both the build and QA process.