What are the responsibilities and job description for the Senior Quality Engineer – AI/LLM Test Automation position at Tek Inspirations LLC?
Position : Senior Quality Engineer – AI/LLM Test Automation
Location: Birmingham, AL
Type: Full-Time
Must Have
- 5 years of QA / SDET / Test Automation experience.
- 1–2 years of hands-on experience testing AI/LLM applications.
- Hands-on experience with LLM evaluation / AI evaluation.
- Experience creating or working with evaluation datasets and ground truth.
- Understanding of LLM judges / AI-as-a-judge, evaluators, scoring, and quality thresholds.
- Experience testing nondeterministic AI-generated outputs.
- Hands-on LangSmith experience.
- Hands-on LangGraph or agent workflow experience.
- Strong Playwright experience, preferably with TypeScript.
- Strong Python and/or TypeScript skills.
- API, integration, regression, and end-to-end testing experience.
- Experience integrating automated tests into CI/CD.
- Strong understanding of Git and Agile development.
Ideal Background
Someone who has worked as an SDET / Senior QA Automation Engineer / AI Test Engineer / AI Quality Engineer and can explain, with real examples:
Production/User Behavior ? Evaluation Dataset ? Ground Truth ? Evaluator/LLM Judge ? Experiment ? Score/Threshold ? Regression ? CI/CD
The candidate should be able to explain how they tested AI outputs, not simply how they used AI tools to write traditional automation tests.
Salary : $120,000