What are the responsibilities and job description for the Quality Engineering Lead position at Jobs via Dice?
Dice is the leading career destination for tech experts at every stage of their careers. Our client, Stanley David and Associates, is seeking the following. Apply via Dice today!
Must Have Technical/Functional Skills
Must Have Technical/Functional Skills
- AI/ML/LLM Quality Engineering experience (testing, validation, evaluation frameworks)
- Expertise in AI evaluation methods (deterministic & non-deterministic systems)
- Validation frameworks, golden datasets, and test harness
- Scalable testing strategies for enterprise AI platforms
- Experience embedding quality practices into AI SDLC / delivery lifecycle
- Model evaluation, monitoring, and continuous quality improvement
- Agent/LLM evaluation and autonomy threshold
- Cross-functional collaboration with engineering, data, and architecture ________________________________________
- Build the AI QA function as a dedicated capability within the broader AI delivery organization.
- Define quality engineering practices for AI systems, including evaluation methods for deterministic and non-deterministic outputs.
- Develop, adopt AI for QE capabilities and improve outcomes of the Quality Engineering function using new AI capabilities
- Establish validation criteria, golden datasets, and test harnesses to support safe deployment and continuous model or prompt changes.
- Work with engineering and synthetic data teams to ensure quality controls are embedded into the AI delivery lifecycle.
- Establish baseline KPIs for QE function and incorporate mechanism to track maturity
- Help define thresholds for agent maturity and autonomy, including when systems can move from analysis to decision and action under appropriate oversight.