Lead Experimentation Analyst

Our client is actively seeking a Lead Experimentation Analyst to join their team! In this role you will utilize data and experimentation to drive smarter product decisions across various teams. The ideal candidate will be someone who understands experimentation deeply but can also turn complex statistical results into simple, actionable recommendations.

Must be able to work fully onsite in their office in San Francisco

Pay: $66.66-76.66/hr

What You’ll Do

• Partner with Product and UX teams to develop testing ideas, hypotheses, and success metrics
• Design and manage A/B and multivariate tests such as audience allocation, sample sizing, test duration, KPIs, and key metrics
• Analyze experiment results and determine whether findings are statistically meaningful
• Perform statistical power and minimum detectable effect analysis
• Identify potential issues with experiments, such as tracking problems, traffic imbalances, or biased results
• Analyze results across different customer segments, devices, and product categories
• Apply advanced experimentation and statistical techniques when appropriate
• Help establish experimentation standards and best practices across product teams
• Work with Analytics and Engineering teams to ensure accurate experiment tracking and data quality
• Present findings and provide clear launch, iterate, or stop recommendations to Product and senior leaders

What We’re Looking For

• Strong background in statistics, experimentation, and data analysis
• Hands-on experience designing and analyzing digital A/B tests
• Experience with statistical methods such as power analysis, variance reduction, sequential testing, or treatment-effect analysis
• Ability to identify experiment quality issues and potential sources of bias
• Strong analytical and problem-solving skills
• Ability to explain complex data and statistical findings in a clear, business-friendly way
• Experience partnering with Product, UX, Analytics, and Engineering teams
• Strong communication skills and the ability to influence product decisions through data

If interested, please submit your resume for consideration.

You can use WorkGrades to collect and manage your references for free and share them with us or anyone else you choose by visiting workgrades.com/home/candidate. Candidates with references are always preferred by our clients. Now is the most important time to stand out from the crowd. We suggest that you ensure you have updated your LinkedIn profile and that you start collecting your references early.

We will consider qualified candidates with criminal histories in a manner consistent with the requirements of the San Francisco Fair Chance Ordinance. California applicants, please view our Privacy Notice here: https://careergroupcompanies.com/california-privacy-notice/ .