Analyse
Grow your serverA/B testing ideas

Grow your server

A/B testing ideas

Ten easy A/B tests you can run this week. Each one is simple to set up and gives a clear winner.

Running A/B tests is easier than you think. Each of these tests can be set up in under an hour and ran for a week. All of them are proven to have moved a real number on a real server.

Use them in order. The first three give the biggest lift on most servers.

1. New spawn vs old spawn

  • Variant A: current spawn
  • Variant B: new spawn you've been wanting to try
  • Win metric: D1 retention

Why it works: the first thing a player sees shapes everything. Small changes (more signs, less clutter, a clearer path to the tutorial NPC) regularly move D1 by 3-5 points.

2. Free kit vs paid cheap rank for newbies

  • Variant A: a free "Starter" kit on first join
  • Variant B: a \$1 rank promoted in the first join message
  • Win metric: 30-day revenue per player

Free stuff boosts engagement. Cheap paid things boost lifetime commitment. This test tells you which effect is bigger on your audience.

3. Store popup timing

  • Variant A: store popup appears on first join
  • Variant B: store popup appears after 10 minutes of playtime
  • Win metric: first-purchase conversion

Pushing the store too early makes a pay-to-play impression. Pushing too late and they forget you sell anything. This test nails the timing for your audience.

4. Join message style

  • Variant A: "Welcome to YourServer!"
  • Variant B: "Welcome %player%! Type /help to get started."
  • Win metric: tutorial started in first session

Personalized and action-driven beats generic. The delta is usually small but free to implement.

5. PvP on at spawn vs PvP off

  • Variant A: PvP zone starts 50 blocks from spawn
  • Variant B: PvP zone starts 200 blocks from spawn
  • Win metric: D1 retention on first-time Java players

If your server allows PvP near spawn, a lot of your retention loss is new players getting farmed. Moving the PvP zone back fixes a lot without changing the core gameplay.

6. Bedrock onboarding path

  • Variant A: same tutorial as Java
  • Variant B: simplified tutorial specifically for Bedrock
  • Win metric: D1 retention for Platform = Bedrock

Bedrock players have different controls and different expectations. A tutorial designed for Java often loses them. Split tutorials can double Bedrock retention.

7. Rank names

  • Variant A: "VIP", "VIP+", "MVP" (tier-style)
  • Variant B: "Pioneer", "Voyager", "Captain" (theme-style)
  • Win metric: rank sales revenue

Tier names feel pay-to-win. Themed names feel like they belong to the server. Revenue often climbs with themed names without changing the actual perks.

8. Daily reward size

  • Variant A: 50 coins per day
  • Variant B: 25 coins day 1, 50 day 2, 100 day 3 (escalating)
  • Win metric: D7 retention

Humans love a streak. Escalating rewards beat flat rewards for come-back rate.

9. First-quest difficulty

  • Variant A: "Gather 10 wood" (easy)
  • Variant B: "Defeat the spawn golem" (moderate)
  • Win metric: quest completion rate, then session length

Too easy and the quest feels meaningless. Too hard and they quit. This test finds the sweet spot.

10. Discord promotion copy

  • Variant A: "Join our Discord"
  • Variant B: "Join 10,000 players on our Discord"
  • Win metric: Discord join rate (track via custom event)

Social proof is nearly free to add and always helps.

How to run any of these

  1. Create the test in Analyse (A/B tests)
  2. Implement both variants in your plugin (SDK: A/B tests)
  3. Launch and wait for 95% statistical confidence (anywhere from 3 days to 3 weeks depending on traffic)
  4. Ship the winner, retire the loser, pick the next test

Commit to one a month

You don't need to run all ten. Pick one test per month, actually finish it, and stack the wins. Twelve successful tests a year is a massively improved server.

Related