Create experiment

11 min

Create experiment

// ---------------------------------------------------------------------------- //
//                              Create experiment.                              //
// ---------------------------------------------------------------------------- //

// Hypothesis – Showing the number of items left in stock on the products detail page (PDP) page will increase add to cart.
// Goal Metric(s) – add to cart rate.
// Audience(s) – Everyone
// Ramp Percentage – 100%
// Metrics – add_to_cart
// Distribution Mode –  manual
// Variations – off (33%), On Show Amounts (33%), On Show Amounts Red (33%)

PATCH {{flagsUrl}}/projects/{{projectId}}/flags/{{flagKey}}/environments/{{environment}}/ruleset
Content-Type: application/json
Authorization: {{token}}

[
    {
        "op": "add",
        "path": "/rules/stock_message_test",
        "value": {
            "key": "stock_message_test",
            "name": "Stock Message Test",
            "type": "a/b",
            "distribution_mode": "manual",
            "percentage_included": 10000,
            "metrics": [
                {
                    "aggregator": "unique",
                    "display_title": "add_to_cart",
                    "event_id": {{eventId}},
                    "event_type": "custom",
                    "scope": "visitor",
                    "winning_direction": "increasing"
                }
            ],
            "variations": {
                "off": {
                    "key": "off",
                    "name": "Off",
                    "percentage_included": 3333
                },
                "on_show_amounts": {
                    "key": "on_show_amounts",
                    "name": "On Show Amounts",
                    "percentage_included": 3333
                },
                "on_show_amounts_red": {
                    "key": "on_show_amounts_red",
                    "name": "On Show Amounts Red",
                    "percentage_included": 3334
                }
            },
            "audience_conditions": []
        }
    },
    {
        "op": "add",
        "path": "/rule_priorities/0",
        "value": "stock_message_test"
    }
]

Create an A/B test to experiment if showing the number of items left in stock on the product detail page (PDP) will increase add-to-cart clicks using the Update the Ruleset for a Flag in an Environment API endpoint.

Prerequisite

Before running this request, you must follow the instructions to configure your REST API cookbook.

Create a file

Create a file named create-experiment.rest and copy and paste the recipe into the file.

Experiment info

General information about the A/B test.

// Hypothesis – Showing the number of items left in stock on the products detail page (PDP) page will increase add to cart.
// Goal Metric(s) – add to cart rate.
// Audience(s) – Everyone
// Ramp Percentage – 100%
// Metrics – add_to_cart
// Distribution Mode –  manual
// Variations – off (33%), On Show Amounts (33%), On Show Amounts Red (33%)

Configure API

{{flagsUrl}}, {{projectId}}, {{flagKey}}, {{environment}}, and {{token}} are defined in the settings.json file. Ensure you have configured your Visual Studio Code project correctly. See Configure the Visual Studio Code REST Client.

PATCH {{flagsUrl}}/projects/{{projectId}}/flags/{{flagKey}}/environments/{{environment}}/ruleset
Content-Type: application/json
Authorization: {{token}}

Add first item

op (operation) – Doing an Add.

path – A JSON-pointer to the resource or the resource attribute.

value – The value to be used within the operations.

{
    "op": "add",
    "path": "/rules/stock_message_test",
    "value": { }
}

Configure experiment

key – Key for experiment.

name – Friendly name for the experiment.

type – Type of experiment, a/b test.

distribution_mode – Proportion of traffic sent to a particular variation.

"key": "stock_message_test",
"name": "Stock Message Test",
"type": "a/b",
"distribution_mode": "manual",
"percentage_included": 10000,

Add metric

Adds a metric named add_to_cart that measures the increase of unique conversions per visitor.

Replace {{eventId}} with your event's ID.

"metrics": [
    {
        "aggregator": "unique",
        "display_title": "add_to_cart",
        "event_id": {{eventId}},
        "event_type": "custom",
        "scope": "visitor",
        "winning_direction": "increasing"
    }
],

Add variations

Add three variations that have equal traffic distribution (~33%):

on_show_amounts

on_show_amounts_red

"variations": {
    "off": {
        "key": "off",
        "name": "Off",
        "percentage_included": 3333
    },
    "on_show_amounts": {
        "key": "on_show_amounts",
        "name": "On Show Amounts",
        "percentage_included": 3333
    },
    "on_show_amounts_red": {
        "key": "on_show_amounts_red",
        "name": "On Show Amounts Red",
        "percentage_included": 3334
    }
},

Audience conditions

Optional audience conditions.

"audience_conditions": []

Add second item

The second operation adds the new rule to the flag's rule priority list at position 0, so stock_message_test is evaluated first.

{
    "op": "add",
    "path": "/rule_priorities/0",
    "value": "stock_message_test"
}

Send request

Click Send Request in Visual Studio Code. See the REST Client's documentation on usage for information on how to send requests using the REST Client extension.

Your A/B test rule should look like the following in the UI: