Product Manager, A/B Testing, Product Analytics & Experimentation

Dice is the leading career destination for tech experts at every stage of their careers. Our client, Asterism IT Solutions, is seeking the following. Apply via Dice today!

Title: Product Manager, A/B Testing, Product Analytics & Experimentation

Location: San Francisco, CA 94133 (4 Days onsite a week)

Duration: 6 Months Contract

TN

Video Interview

Job Description

Main keywords for the ideal candidate would be:

• Experimentation
• Key word is RAPID experimentation
• A/B Testing
• Needs to have very heavy experience here
• Retail/Ecommerce
  
  

Experience

Looking for a Sr. product leader who''s well versed in site experience & leading fast-paced experimentations. Rather than a data product manager, this candidate is expected to rapidly test & improve the customer service experience when visiting the websites of WSI & it''s brands.

While other PDMs build product features & work with fullstack engineering team to build hard-end code, this role is about building fairly lightweight concepts through front end that can be tested/validated quickly & moved to production (example: identify drop off spikes at checkout stage, investigate, hypothesize, vet ideas quickly & launch tests).

"Rapid" testing is the key word here... This team runs ~15 monthly tests across 4 major brands. Of those tests, at least 50% are expected to be seen as statistical wins. Candidates will be expected to showcase their previous roles in similar fast-paced environments.

Required Skills

• 5 years of Product Management experience in technology organizations driving requirements gathering & feature definitions
• In-depth experience leading experimentation programs around ecommerce customer experience improvements in a high-volume environment.
• Core ecommerce experience is required (ideally at scale with WSI, not required).
• Experience with Adobe Target, or another AB testing tool (i.e. Optimizely)
• History of successfully partnering with UI/UX teams to understand site/customer experience & build PoCs
• History of successfully partnering with engineering teams to write requirements
• Strong communication skills - consistent experience driving meetings with leadership involvement & command that audience
• Understanding of AB testing process and baseline understanding of statistical concepts in AB test analysis (understanding Statistical significance, confidence intervals, and Sample Size requirements)
• You have strong analytical abilities and interest to use data and metrics to back up recommendations and drive actions.
• Experience working within an Agile environment, with Jira & Confluence
  
  

Day-to-day

• Drive the strategic vision and roadmap of the experimentation program for one of Williams-Sonoma''s portfolio brands to increase company ROI. Collaborate with a cross functional team of product analysts, brand partners, UX designers and engineers to prioritize and execute the most impactful work.
• Lead the definition, prioritization and scoping of test experiences for engineering team. Craft business requirements and user stories and to support our testing engineering teams through solution design, development, QA, and implementation phases of delivery.
• Ensure each test idea has a business opportunity driven by data, a clear audience definition, measurable KPIs, an execution plan, and stakeholder buy in.
• Move fast in an agile and iterative team environment.
• Serve as liaison between brand and technology teams to ensure testing experiences meet expectations and provide customer value.
• Socialize a clear view of priorities and progress against experimentation roadmap with business partners and senior leadership.
• Work with your Product Analytics partner to define KPIs and metrics for tests, monitor test performance, and review postmortem results & recommendations. Drive action based on test results that align with the business goals, including development of future test ideas.
• Identify testing opportunities using test results analysis and other touchpoints (UX studies, analytics reporting, market and industry trends, competitor product analysis. etc.). Collaborate with brand partners to ideate on testing opportunities to maintain a full pipeline of experiments for engineering.
• Create and execute test campaigns using Adobe Target and homegrown A/B testing solution, including audience creation, enlisting IT development resources for more complicated functional test treatments.
• Develop and document testing processes and best practices to further improve campaign creation quality and efficiency.
• Work with the rest of the Product Analytics & Experimentation team to foster a culture of data-driven decision making across e-commerce that encourages validating or disproving assumptions about how we believe customers may react to various site experience scenarios.