What are the responsibilities and job description for the Senior AI Data Analyst position at McKinley Marketing Partners?
Job Description
Our retail client is seeking a Senior AI Data Analyst to support their rapidly growing Digital organization. You will analyze data flowing into and out of our machine learning models, with a strong focus on model input quality, model performance, experimentation, and post-deployment monitoring. You will design and build analytical frameworks and visualizations that help the business understand how AI features are performing in real time and where to invest next.
You will primarily work within the Databricks ecosystem (Azure preferred), using Python, PySpark, and SQL to build robust, scalable analytics. You will be responsible for telling the story behind the data through dashboards, visualizations, and written insights that drive decisions across Digital, Product Management, Marketing, and Operations.
Responsibilities
Our retail client is seeking a Senior AI Data Analyst to support their rapidly growing Digital organization. You will analyze data flowing into and out of our machine learning models, with a strong focus on model input quality, model performance, experimentation, and post-deployment monitoring. You will design and build analytical frameworks and visualizations that help the business understand how AI features are performing in real time and where to invest next.
You will primarily work within the Databricks ecosystem (Azure preferred), using Python, PySpark, and SQL to build robust, scalable analytics. You will be responsible for telling the story behind the data through dashboards, visualizations, and written insights that drive decisions across Digital, Product Management, Marketing, and Operations.
Responsibilities
- Research, prototype, and build analytics solutions and visualizations to support the AI/ML pipeline, from data ingestion through model deployment.
- Analyze training and production data for machine learning models to identify data quality issues, drift, feature importance, and behavioral shifts.
- Design and execute robust experimental frameworks (A/B tests, holdout tests) to quantify the impact of AI-driven features and digital product changes.
- Develop clear KPI definitions and measurement strategies for AI features and digital initiatives, ensuring consistency across teams.
- Create and maintain dashboards and visual reports in Python and Power BI that communicate trends, anomalies, and opportunities to business stakeholders.
- Collaborate closely with data scientists, ML engineers, product managers, and digital leaders to translate complex analytical findings into actionable recommendations.
- Continuously refine analytical models and reporting as new data sources, features, and products are introduced.
- Stay current with emerging tools, libraries, and techniques in analytics, data engineering, and AI observability, and introduce best practices into the team.
- Ensure timely delivery of high-quality analysis and documentation in a fast-paced, agile environment.
- Strong knowledge of statistical techniques and advanced mathematics (e.g., hypothesis testing, regression, experimental design, time-series analysis).
- 3 years of experience as a data analyst, data engineer, or data scientist working within the Databricks ecosystem (Azure Databricks preferred).
- 5 years of experience applying statistical techniques to analyze, segment, and visualize data, with hands-on experience in experimental design, KPI calculation, and A/B testing.
- 4 years of experience manipulating and analyzing large-scale datasets using Python, PySpark, and/or SQL.
- Expert-level experience with data visualization tools and libraries (e.g., Python data viz stack, Power BI) to create interactive dashboards and compelling visual narratives.
- Bachelor’s degree in Computer Science, Engineering, Mathematics, Statistics, or related field; or equivalent practical experience.