Between-subjects experiments

An A/B test here is a between-subjects experiment: each participant is randomly assigned to one of two conditions and sees one version of a stimulus, then everyone answers the same outcome measures. You compare the arms on those measures. The fastest way to start is the A/B test starter template on Explore — a real two-condition design you edit and run.

​

How the starter is built

The A/B starter is a genuine two-arm design, not a cosmetic one. It ships with:

• Two conditions — Version A and Version B — seeded as random-assignment arms on the version.
• Two stimulus screens, each gated to one condition: the Version A screen shows only to the version-a arm, the Version B screen only to the version-b arm. Each holds placeholder wording you replace with the two messages you want to compare.
• Shared outcome measures every participant answers regardless of arm: a 7-point appeal rating, a share-intention question, and an attention check.
• A welcome screen, consent, and a thank-you screen.

​

Editing it for your study

​

Random assignment

Two plain-language controls sit above your conditions in the Builder’s Conditions panel, and together they decide how arriving participants are split across the arms. Group sizes — how big each arm should be relative to the others:

• Equal — every arm the same size. The panel shows each arm’s expected share (e.g. ≈50% for two arms).
• Custom — set each arm’s share as a percentage. Use this to skew the split deliberately (e.g. send 20% to a pilot arm), or for any non-even design. A running total nudges you when the shares don’t add up to 100%.

Randomization — how each participant is placed into an arm:

• Simple random (default) — each participant is drawn independently by weighted random assignment over the group sizes. With equal sizes that’s a coin flip per person, so the split is even in expectation, not a hard quota. Natural sampling variance means the arms won’t always end up exactly equal — the smaller your sample, the larger the wobble.
• Balanced — the app keeps the arms even as participants arrive, using permuted-block assignment: it fills randomly-ordered blocks sized to your ratio (equal two-arm → blocks of 2; a 1:2 split → blocks of 3), so every time a block completes the arms are back in exact proportion. The order within each block is still random, so assignment stays unpredictable for any individual participant. The panel tells you the block size (“evens out every N participants”).

​

Beyond two arms — conditional visibility

An A/B test assigns participants to fixed conditions upfront, but you can also gate blocks dynamically based on earlier responses. Conditional visibility lets you show or hide any block depending on what a participant answered in a previous block — without needing separate conditions. For example:

• If a participant answers “yes” to a question, show them a follow-up; if “no”, skip it.
• If they score above a threshold on a pretest, show the advanced version; below it, show the simplified one.
• If they click a specific region of an image, reveal a detail block; otherwise hide it.

To set this up, open a block in the Builder and scroll to Show this block when. Pick an answerable block from earlier in the flow, choose an operator (equals, contains, answered, between, etc.), and set the condition value. Multiple conditions can be combined with AND (all must be true) or OR (any must be true). Conditional visibility works within any design — a single-arm study, an A/B test, or a factorial design. It’s especially useful for adaptive flows where the next step depends on what participants tell you.

​

Factorial designs with variants

Beyond simple A/B, you can test multiple factors at once — factorial designs where participants are split across all combinations of levels. If you’re comparing tone × length × image style, that’s 3 factors that might produce 2×2×2 = 8 cells. In the Builder, open the Variants section to define factors (e.g. “Tone”, “Length”) and their levels (e.g. “Formal” / “Casual” for Tone, “Short” / “Long” for Length). Then bind fields from your blocks — perhaps the message text, an image, or a number — to each factor. Each time a field varies, you specify its value for every level. When the study runs, each participant is randomly assigned one combination — one level from each factor. The exported dataset includes a variant_combination column so you can compare outcomes across cells. Shared content (blocks, measures) stays the same; only the bound fields change. Variants work independently of conditions — you can have variants and conditions in the same study, giving you a hybrid design (factors nested inside or crossed with arms).

​

Recruiting a balanced sample on Prolific

From the Run stage, connect Prolific and create a study with a single target N. Prolific recruits that many participants to your one recruitment link; the app then assigns each arriving participant to a condition using the Randomization method you chose above. The Running view flags a study as imbalanced when the gap between the smallest and largest arm exceeds 20% of the largest — most useful under Simple random, where drift is expected; a Balanced study should rarely trip it.

​

Reading results by condition

The study Results break down per condition: each arm’s name and its completed-response count, plus per-question summaries (means for numeric items, option counts for categorical, counts for text). The exported dataset tags every response with the condition the participant was assigned, so you can compare the arms directly in your own analysis.