A/B testing for React and Gatsby

12 min

A challenge when running A/B tests on React applications is ensuring reliable selectors that do not break when the underlying HTML or application structure changes. Dynamic selectors (such as those derived from component libraries or generated class names) can have inconsistencies, making experimentation difficult and time-consuming. Additionally, relying on position-based selectors like :nth-child() or XPath-based selection methods leads to configurations that fail with minor HTML changes.

This issue is not unique to Optimizely Web Experimentation. Any web-based experimentation tool that applies changes with CSS selectors faces similar difficulties if selectors are not stable. Although Optimizely provides functionality to work around dynamic selectors, a proactive approach to creating stable, consistent selectors saves time and improves long-term reliability. See Target dynamic selectors for information.

Why stable selectors matter

Reduced maintenance – Stable, meaningful selectors reduce the amount of rework needed when the UI changes.

Increased reliability – When CSS selectors are stable, experiments do not break from minor HTML reordering or updated class structures.

Scalability – By standardizing selector strategies, you can scale experimentation efforts without extensive developer intervention for every new test.

CSS selectors

CSS selectors style HTML elements. Experimentation tools rely on them to identify which elements to modify, hide, or manipulate. Common types of selectors include the following:

Class selectors (.)

Example – <div class="product-card">Product 1</div> <div class="product-card">Product 2</div>

Target – .product-card

ID selectors (#)

Example – <div id="header-banner">Header Banner</div>

Target – #header-banner

Attribute selectors ([])

Example – <button data-test-id="add-to-cart">Add to Cart</button>

Target – [data-test-id="add-to-cart"]

Attributes like data-test-id, aria-*, or data-experiment can provide stable, meaningful hooks for experiments.

Element selectors (such as div or button)

These selectors are too broad for most testing and are more susceptible to unintended changes.

Combinators and pseudo-classes (such as :nth-child())

Example –

<ul>
  <li>Item 1</li>
	<li>Item 2</li>
</ul>

Target – ul li:nth-child(2)

However, this targeting breaks easily if the structure changes.

Use unique static attributes

Instead of relying on dynamically generated or position-based selectors, add stable attributes that remain consistent across builds and changes.

Existing aria or data attributes

If elements already have aria-* or data-* attributes, use them. These attributes often exist for accessibility or application logic and are less likely to change.

Example – <button aria-label="Add to Cart" class="add-to-cart-button">Add to Cart</button>

Stable target – [aria-label="Add to Cart"]

Experiment-specific attributes

Add static attributes like data-experiment="variantA" to key elements.

Example – <div data-experiment="variantA" class="product-banner">Product Banner</div>

Target – [data-experiment="variantA"]

Custom attributes

Introduce data-test-id="unique-element" attributes to critical elements. You can add these directly in the component's JSX.

By tagging important elements with stable attributes, experiments remain robust even if the surrounding structure changes.

Example (React component) –

import React from "react";

const ProductCard = ({ productName }) => {
  return (
    <div data-test-id="product-card" className="product-card"><h3>{productName}</h3></div>
  );
};

export default ProductCard;

Target – [data-test-id="product-card"]

Gatsby implementation

To minimize manual work, you can add these attributes using build-time scripts or as part of the component logic. In Gatsby, you could use the onCreateNode or onPreRenderHTML API to add attributes programmatically during the build process.

import React from "react";
 
const ProductCard = ({ productName }) => {
  return (
    <div data-test-id="product-card" className="product-card">
      <h3>{productName}</h3>
    </div>
  );
};
 
export default ProductCard;

This ensures all ProductCard components have the same data-test-id, making them easy to target.

Automate adding data attributes

Consider automating the process if manually adding attributes to every component is cumbersome.

Target areas – Work with stakeholders to identify the parts of the application that are most likely to be tested. This limits the scope and reduces development work.

Build-time scripts – Use your build pipeline (Webpack, Rollup, or other tools) or Gatsby's build-time APIs to inject data-test-id attributes into rendered elements automatically.

Server-side rendering or pre-rendering steps – If your React application uses server-side rendering (SSR), you can parse the HTML before sending it to the browser and insert stable attributes dynamically based on predefined rules.

Reduced developer involvement – Automation ensures consistent application of these attributes without requiring developers to manually edit each component.

Standardize class naming with a component library

Adopting a clear, consistent naming convention for component library classes can simplify experiment targeting.

Reusable classes – Define standardized classes for UI components like btn-primary, nav-link, product-card, and hero-banner. Adding these standardized classes during the component development stage ensures elements can be targeted reliably during experiments.

Component prop enhancements – Extend components to accept props that append experimental classes or attributes for quick tagging without component code modification. This makes the classes available without modifying individual component files repeatedly.

By establishing a naming pattern, marketers can quickly identify target elements, and developers have a predictable system for reference.

Use attribute decorators or custom hooks in React

React, including Gatsby, lets you create custom hooks or higher-order components that systematically add testing attributes.

React example

import { useEffect, useRef } from 'react';

const useExperimentTag = (tag) => {
  const ref = useRef(null);
  
  useEffect(() => {
    if (ref.current) {
      ref.current.setAttribute('data-test-id', tag);
    }
  }, [tag]);
  
  return ref;
 
};

// Usage in a component
import React from 'react';

const ProductCard = ({ productName }) => {
  const cardRef = useExperimentTag('product-card');
  
  return (
    <div ref={cardRef} className="product-card">
    	<h3>{productName}</h3>
		</div>
	);
};

export default ProductCard;

Such a hook ensures all instances of ProductCard have a stable identifier without individually adding the attribute in every JSX element.

Gatsby example

import { useEffect } from 'react';
 
const useExperimentTag = (ref, tag) => {
  useEffect(() => {
    if (ref.current) {
      ref.current.setAttribute('data-test-id', tag);
    }
  }, [ref, tag]);
};
 
// Usage in a Gatsby component
import React, { useRef } from 'react';
 
const ProductCard = ({ productName }) => {
  const cardRef = useRef(null);
  useExperimentTag(cardRef, 'product-card');
 
  return (
    <div ref={cardRef} className="product-card">
      <h3>{productName}</h3>
    </div>
  );
};
 
export default ProductCard;

This ensures all targeted components have consistent attributes, making them easier to select for experimentation.

Use a DOM parser in the build pipeline

Use a DOM parser in the build pipeline for applications where HTML files are generated, such as with SSR or static site generation.

This lets you inject attributes without touching the React codebase repeatedly, ensuring consistent targeting across all builds. It can also run during the Gatsby build process to programmatically add consistent classes or attributes to specific elements.

Configuration file for attributes

Create a JSON or YAML file specifying which elements should receive which attributes.

{
  "elements": [
    {
      "selector": "div.product-card",
      "attributes": {
        "data-test-id": "product-card"
      }
    },
    {
      "selector": "button.add-to-cart",
      "attributes": {
        "data-test-id": "add-to-cart-button"
      }
    }
  ]
}

DOM injection script

Use jsdom or another parser in a Node.js script that runs after the build.

const fs = require('fs');
const jsdom = require('jsdom');
const { JSDOM } = jsdom;

const config = require('./config.json');

fs.readFile('public/index.html', 'utf8', (err, data) => {
  if (err) throw err;
	const dom = new JSDOM(data);
	const document = dom.window.document;

	config.elements.forEach((element) => {
		const nodes = document.querySelectorAll(element.selector); 
    nodes.forEach((node) => {
			Object.entries(element.attributes).forEach(([key, value]) => { 
        node.setAttribute(key, value);
			});
		});
	});


	fs.writeFile('public/index.html', dom.serialize(), (err) => { 
    if (err) throw err;
		console.log('Attributes injected successfully');
	});
});

Gatsby example.

const fs = require('fs');
const jsdom = require('jsdom');
const { JSDOM } = jsdom;
 
const config = require('./config.json');
 
exports.onPostBuild = ({ reporter }) => {
  fs.readFile('public/index.html', 'utf8', (err, data) => {
    if (err) {
      reporter.panicOnBuild(`Error reading HTML file`, err);
      return;
    }
 
    const dom = new JSDOM(data);
    const document = dom.window.document;
 
    config.elements.forEach((element) => {
      const nodes = document.querySelectorAll(element.selector);
      nodes.forEach((node) => {
        Object.entries(element.attributes).forEach(([key, value]) => {
          node.setAttribute(key, value);
        });
      });
    });
    fs.writeFile('public/index.html', dom.serialize(), (err) => {
      if (err) {
        reporter.panicOnBuild(`Error writing HTML file`, err);
      } else {
        reporter.info('Attributes injected successfully');
      }
    });
  });
};

Define a clear testing zone

If most experiments occur in a particular section of the site, consider marking that area clearly. This helps both developers and marketers quickly identify where experiments can be safely conducted.

<div data-testing-zone="true">
  <!-- Elements that are commonly tested -->
	<div data-test-id="product-card" class="product-card">Product A</div>
	<div data-test-id="add-to-cart-button" class="add-to-cart">Add to Cart</div>
</div>

Use an iterative approach to tagging elements

Don’t feel pressured to tag every element at once. Start with high-impact areas and gradually expand as testing needs grow.

Begin with key elements – Focus on conversion buttons, top banners, product listing areas, or forms first.

Scale over time – Add stable attributes to secondary elements as you need more experiments to avoid overwhelming developers and improve your experimentation environment.

Educate and align

Ensuring buy-in from your development, marketing, and product teams is crucial.

Educate the team – Explain how stable selectors save developer time, reduce experiment setup complexities, and increase test reliability.

Show quick wins – Highlight instances where stable selectors helped run successful experiments faster or avoided breaks during minor UI updates. Examples help other stakeholders see the value of these practices.

By emphasizing the long-term benefits, you can encourage consistent adoption and maintenance of stable attributes throughout the codebase.