home / Marketing Skills / A/B Testing Skill by @coreyhaines31

A/B Testing Skill by @coreyhaines31

FREE

github stars★ 50,667updatedMay 2026repo@coreyhaines31

activity (commits, last 6 months)

6mo ago0

this month0

$git clone https://github.com/coreyhaines31/marketingskills

The skill is the skills/ab-testing folder inside that repository. Copy it into the folder your agent loads skills from; check your agent's documentation for where that is.

about A/B Testing Skill by @coreyhaines31

Plan, design, or implement an A/B test or experiment, or build a growth experimentation program.

Plan and design A/B tests with a clear hypothesis framework
Covers test types, sample size, and metrics selection
Guides variant design and building a growth experimentation program

our take

Plans and runs A/B tests properly. It begins with an assessment, then enforces principles: start from a hypothesis, test one thing, apply statistical rigour and measure what matters.

when to use it

You are about to launch a split test and need a plan
You want sample size and significance handled correctly
You are building an experimentation programme

before you install

Small sites often lack the traffic for reliable results, so check sample size first
Stop tests at the planned end, not when results look good

what's inside

The original SKILL.md is organised into these sections:

Initial Assessment
Core Principles
1. Start with a Hypothesis
2. Test One Thing
3. Statistical Rigor
4. Measure What Matters
Hypothesis Framework
Structure
Example
Test Types
Sample Size
Quick Reference

source and license

Written and maintained by @coreyhaines31. It is published under the MIT license, which allows reuse with attribution. SkillDirectory does not claim authorship: this page adds our own review and links back to the original, and the skill's text belongs to its author.

View the full SKILL.md on GitHub ↗

Read the opening of the original SKILL.md

# A/B Test Setup

You are an expert in experimentation and A/B testing. Your goal is to help design tests that produce statistically valid, actionable results.

## Initial Assessment

**Check for product marketing context first:**
If `.agents/product-marketing.md` exists (or `.claude/product-marketing.md`, or the legacy `product-marketing-context.md` filename, in older setups), read it before asking questions. Use that context and only ask for information not already covered or specific to this task.

Before designing a test, understand:

1. **Test Context** - What are you trying to improve? What change are you considering?
2. **Current State** - Baseline conversion rate? Current traffic volume?
3. **Constraints** - Technical complexity? Timeline? Tools available?

---

## Core Principles

### 1. Start with a Hypothesis
- Not just "let's see what happens"
- Specific prediction of outcome
- Based on reasoning or data

### 2. Test One Thing
- Single variable per test
…

Excerpt only, quoted from the author's repository. Open the repository for the complete file.

↓ download .zip