A/B test a link: split traffic by weight

A/B test a short link: split its visitors between pages by weight, each visitor kept on one side for the day, then compare clicks and events per variant.

On this page
  1. How A/B testing a short link works
  2. Set up an A/B test on a short link
  3. Test one audience: the link's split or a rule's
  4. Read the A/B test results
  5. Change or end a test
  6. Common questions
  7. Next steps

To A/B test a link, you add two or more variants to a short link, each with its own page and a weight, and Brevily splits the link's visitors between them. Each visitor stays on the same variant for the day, and the A/B test panel shows clicks, visitors and events for each one.

Say your newsletter goes out on Tuesday with one button, and you can't decide between two landing pages. Put the short link on the button, send half the readers to each page, and let the numbers choose. The link never changes, so it can go out before the test starts and keep working after it ends.

Before you start: you need a short link; see how to create a short link. A/B tests depend on your plan; see pricing. To compare sign-ups or sales, turn on Track events in the link's Settings tab and report them; see link tags and events.

  • Variants: each has a name, a destination and a weight.
  • Weights are shares, not percentages. Two variants at 70 and 30 send 70% and 30% of visitors; three at 1, 1 and 2 send 25%, 25% and 50%. The panel draws the split as a bar.
  • Sticky for the day, with no cookie. Each visitor is placed at random, by weight, and gets the same variant on every visit that day, in your organisation's time zone. The next day they may be placed again.
  • The variants replace the link's destination. To test a new page against your current one, add the current page as a variant too.
  1. Open Links, choose the link and open its Routing tab.
  2. In A/B test, choose Add variant.
  3. Give it a Name, such as "A · Current page", a Weight, 1–1000 (blank means 50), and the Destination.
  4. Choose Add variant.
  5. Add the next variant the same way, such as "B · New page".

[screenshot: the Add a variant dialog with Splits, Name, Weight and Destination]

Once a variant is added, the panel shows Running; add a second for a real test. Each variant you add counts as one destination edit from your plan's allowance, and its destination is checked for safety like the link's own.

On a link with targeting rules, the Add a variant dialog asks what the variant Splits:

  • Everyone the rules don't catch: the link's own split. On a link without rules, that's everyone.
  • One rule's visitors, such as "Phones in India": a test that runs in one market, on one kind of device or in one time window, while everyone else goes where they always did.

A link can run both at once, and each split gets its own bar and table. A split holds up to 10 variants, and a link up to 40. Removing a rule removes its test too; the console asks first.

Read the A/B test results

Each split's table in the A/B test panel shows, per variant:

  • Clicks and Unique visitors, a visitor counted once a day.
  • Events: sign-ups, sales or other events reported against those clicks.
  • Rate: events as a share of clicks, the number to compare between variants.

[screenshot: the A/B test panel with a 50/50 split bar and the results table]

Results count from when the newest variant was added, so every variant is compared over the same days. Rate doesn't depend on the weights, so a variant on 20% of visitors can still win. An event whose click is older than your plan keeps can't be matched to a variant, and the panel counts it apart.

To see single clicks, open the link's Clicks tab: Routed to names the variant, such as "Variant · B · New page". For a rule's split it names the rule, so tell variants apart by the destination beside it.

Change or end a test

A variant can't be edited in place. To change a weight or a page, remove the variant and add a new one with a new name; a removed variant's name stays taken, because its past clicks keep it. Adding or removing a variant re-deals visitors, and adding one restarts the results, so set the test up before the link goes out.

To end a test, remove the losing variants, or remove them all to send everyone back to the link's own destination. While the link's own split has variants, scheduled destination changes reach nobody, and the Scheduled destinations panel says The A/B test comes first.

Common questions

Does a returning visitor always see the same A/B test variant?

Yes, for the rest of the day, on the same browser and network, while the variants stay the same. On another day, device or network, the visitor can be placed in another variant.

Do QR code scans count in a short link's A/B test?

Yes. A scan opens the same short link, so scans are split between the variants like any other visit.

Can I A/B test a short link only on phones, or in one country?

Yes. Add a targeting rule for those visitors on the link's Routing tab, then add variants that split that rule.

What happens to an A/B test if my plan drops A/B tests?

The variants are kept, and the link ignores them until your plan includes A/B tests again. Meanwhile visitors go to the rule's page or the link's own destination.

Next steps

All how-to guides

Brevily is invite-only, for now

Say what you would use it for and a person replies. Already invited? Sign in.