Digital Experimentation, Analyst II

Overview

Discover the possibilities of our progressive, omnichannel approach to beauty retail. At Ulta Beauty, our E-commerce associates sit at the cutting edge of our efforts to reimagine the future of retail. Through a full suite of new and emerging technologies, this team is creating an enhanced buying experience for guests who crave simplicity, choice, personal insights and the joy of discovery. With greater interactivity. Inspired applications of big data, AI and UI/UX. And creative that persuades and delights. All of this, with the opportunity to experience smart ideas come to life.

At Ulta Beauty, we’re forever imagining. Forever striving. Forever investing in next-gen—with deep and rich opportunities for ongoing learning, growth and advancement. We encourage you to take control of your career and set your own ceiling. For curious and big-picture thinkers who can flex to the need and who want to stretch themselves through courage and conviction, Ulta Beauty provides an environment perfectly suited to your ambitions. Let’s build brilliant together.

THE IMPACT YOU CAN HAVE:

The Analyst, Experimentation & Testing plays a key role in advancing our digital experimentation program by delivering high-quality measurement, analysis, and insights across web and app experiences. This role is responsible for validating and analyzing test results, and translating data into actionable recommendations that drive incremental business impact.

The ideal candidate brings strong analytical rigor, curiosity, and storytelling skills, with the ability to distill complex findings into clear narratives for cross-functional teams. This role supports the Manager of Experimentation & Testing in ensuring experimentation insights inform strategic decision-making across digital teams (UX, Digital Experience, Product PODs, Personalization, Content, etc.).

PRINCIPAL DUTIES & RESPONSIBILITIES (*Essential Functions)

• Analyze A/B and multivariate tests using Testing tools (AB Tasty and Adobe Target) and other data sources, ensuring data accuracy, statistical reliability, and clear interpretation of results.
• Validate experiment performance, including traffic allocation, segmentation, audience overlap, etc.
• Build post-test analyses that quantify incremental impact on KPIs, customer behavior, and conversion outcomes.
• Synthesize insights into compelling stories that highlight implications, risks, and recommended next steps.
• Create test readouts, dashboards, and self-service reporting that scale learnings across teams.
• Partner with the Digital Analytics and Strategic Analytics teams to analyze test performance and uncover insights within Customer Journey Analytics (CJA).
• Assist in defining success metrics, analytical requirements, and measurement plans for experimentation initiatives.
• Ensure learnings are cataloged and shared to support a strong organizational repository and test-and-learn culture.
• Support the Manager of Experimentation & Testing in refining analytical frameworks, governance, and best practices.
• Maintain transparency and communication of testing insights across business units.
  
  

THE ESSENTIALS FOR SUCCESS:

• Bachelor’s Degree in Analytics, Statistics, Data Science, Marketing, or related quantitative field.
• 1-3 years of experience in digital analytics, experimentation, or customer behavior analytics; eCommerce or retail experience preferred.
• Experience with A/B Testing Tools - Adobe Target or AB Tasty preferred.
• Familiarity with Adobe Analytics and/or Customer Journey Analytics (CJA) a plus.
• Foundational knowledge in statistical analysis and hypothesis testing.
• Ability to translate complex data into clear, concise, and actionable business insights.
• Proficiency with data visualization, dashboard creation, and developing clear analytical documentation.
• Excellent communication and collaboration skills, with the ability to support both technical and non-technical partners.
• High attention to detail, strong critical thinking, and a natural curiosity about digital customer behavior.
• 1
  
  

The pay range for this position is $66,200.00 - $82,000.00 / Year with the opportunity for eligible associates to earn additional compensation pursuant to the Company’s bonus plan. Exact pay will be based on factors including, but not limited to relevant education, qualifications, certifications, experience, level, shift, geographic location, and business and organizational needs. Full-time positions are eligible for paid time off, health, dental, vision, life and disability benefits. Part-time positions are eligible for dental, vision, life, and disability benefits. For additional information concerning our benefits, visit our Benefits and Career Development page: https://learn.bswift.com/ulta

About

At Ulta Beauty (NASDAQ: ULTA), the possibilities are beautiful. Ulta Beauty is the largest North American beauty retailer and the premier beauty destination for cosmetics, fragrance, skin care products, hair care products and salon services. We bring possibilities to life through the power of beauty each and every day in our stores and online with more than 25,000 products from approximately 500 well-established and emerging beauty brands across all categories and price points, including Ulta Beauty’s own private label. Ulta Beauty also offers a full-service salon in every store featuring—hair, skin, brow, and make-up services.

We will consider for employment all qualified applicants, including those with arrest records, conviction records, or other criminal histories, in a manner consistent with the requirements of any applicable state and local laws, including the City of Los Angeles’ Fair Chance Initiative for Hiring Ordinance, the San Francisco Fair Chance Ordinance, and the New York City Fair Chance Act.