skills sourced live from GitHub · all skills listed are free · each skill is provided under its author's license.
home / Testing & QA / Ab Test Setup Skill by @sickn33
The skill is the plugins/agentic-awesome-skills-claude/skills/ab-test-setup folder inside that repository. Copy it into the folder your agent loads skills from; check your agent's documentation for where that is.
Use when designing an A/B or split test: define the hypothesis, control and variants, estimate sample size, verify tracking.
Designs an A/B test rigorously: define the hypothesis, lock it as a hard gate, check assumptions and validity, set variants and sample size, verify tracking and predeclare metrics and stopping rules.
The original SKILL.md is organised into these sections:
Written and maintained by @sickn33. It is published under the MIT license, which allows reuse with attribution. See the author profile. SkillDirectory does not claim authorship: this page adds our own review and links back to the original, and the skill's text belongs to its author.
# A/B Test Setup ## 1️⃣ Purpose & Scope Define an experiment that can answer a specific product question, and verify its assumptions before exposing users. This procedure cannot guarantee validity by itself. - Documents the stopping rule - Estimates sample needs under stated assumptions - Makes the hypothesis and decision criteria reviewable --- ## 2️⃣ Pre-Requisites You must have: - A clear user problem - Access to an analytics source - Roughly estimated traffic volume ### Hypothesis Quality Checklist A valid hypothesis includes: - Observation or evidence - Single, specific change - Directional expectation - Defined audience - Measurable success criteria --- ## 3️⃣ Hypothesis Lock (Hard Gate) Before designing variants or metrics, you MUST: - Present the **final hypothesis** - Specify: - Target audience - Primary metric - Expected direction of effect - Minimum Detectable Effect (MDE) …
Excerpt only, quoted from the author's repository. Open the repository for the complete file.