skills sourced live from GitHub · all skills listed are free · each skill is provided under its author's license.
home / Testing & QA / Ab Testing Framework Skill by @thatrebeccarae
The skill is the skills/ab-testing-framework folder inside that repository. Copy it into the folder your agent loads skills from; check your agent's documentation for where that is.
A/B and multivariate testing methodology.
An A/B and multivariate testing methodology that is platform-agnostic: hypothesis, sample size calculation, test execution rules, statistical significance and interpreting results while avoiding common pitfalls.
The original SKILL.md is organised into these sections:
Written and maintained by @thatrebeccarae. It is published under the MIT license, which allows reuse with attribution. See the author profile. SkillDirectory does not claim authorship: this page adds our own review and links back to the original, and the skill's text belongs to its author.
# A/B Testing Framework Design, run, and analyze conversion experiments with statistical rigor. ## Install ```bash git clone https://github.com/thatrebeccarae/claude-marketing.git && cp -r claude-marketing/skills/ab-testing-framework ~/.claude/skills/ ``` ## Test Design Process ### Step 1: Hypothesis **Template:** If we [change X], then [metric Y] will [increase/decrease] by [Z%] because [reason]. **Good hypothesis:** "If we change the CTA from Get Started to Start Free Trial, then signup rate will increase by 15% because it reduces uncertainty about cost." **Bad hypothesis:** "If we change the button color, conversions will improve." (No reasoning, no expected magnitude.) ### Step 2: Sample Size Calculation To determine how long to run a test: ``` Required sample per variation = 16 * (p * (1-p)) / (MDE^2) Where: p = baseline conversion rate (as decimal) MDE = minimum detectable effect (as decimal) ``` | Baseline Rate | 10% MDE | 20% MDE | 30% MDE | …
Excerpt only, quoted from the author's repository. Open the repository for the complete file.