AI Test Engineer Mclean, Va (Fully Onsite - local)

Location: Mclean, Va (Fully Onsite - local)

Duration: Long Term 

Interview Information:

• Rounds: 2 Rounds (30-60 Mins Each)
• Interview Type: 1st Round - Virtual | 2nd Round - Onsite

Must-Have Qualifications:

• Must Have 5 Years Of Experience And A Strong Ai Development Background.
• Must Have Hands-On Experience Building Agentic Workflows, Automating Testing Using Ai, And Working With Tools Such As Github, Copilot, And Claude.

Job Description

1) Agentic Test Automation Foundation (Reusable Patterns Reference Implementations)

• Design And Implement Agentic Testing Patterns That Can Be Adopted By Multiple Underwriting Teams (And Later Other Domains).
• Create Reference Implementations (Sample Repos / Templates) Demonstrating:
  • Test Generation Assistance (From Requirements, Apis, Contracts, Schemas)
  • Test Maintenance Assistance (Auto-Updating Selectors/Contracts, Flaky Test Triage)
  • Failure Analysis Assistance (Root Cause Suggestions, Log Correlation, Defect Drafting)
• Establish A Standard Architecture For Test Code Organization, Tagging, Data Management, And Execution Across Ui Api Service Layers.

2) Coverage Standards, Templates, And Governance

• Define And Publish Coverage Standards (What "Good" Looks Like) Including:
  • Minimum Coverage Expectations By Service/Component
  • Test Type Mix (Unit Vs Api Vs Ui Vs Contract Vs Integration)
  • Risk-Based Prioritization And Traceability To Requirements
• Provide Templates Usable Across Teams:
  • Test Plan Templates
  • Test Case/Spec Templates (Gherkin-Style Or Equivalent)
  • Definition Of Ready / Definition Of Done Quality Checklists
• Create A Scalable Tagging/Metadata Strategy (E.G., Feature, Service, Risk, Priority, Data Sensitivity) To Support Reporting And Quality Gates.

3) Genai-Assisted Reporting And Quality Insights Across Microservices

• Build Automated Reporting That Aggregates Test Service Data Across Multiple Microservices, Such As:
  • Test Execution Results (Karate/Playwright Ci Runs)
  • Service Health Signals (Logs/Metrics/Traces If Available)
  • Defect Signals (Issue Tracker Metadata If Available)
• Generate Genai-Driven Summaries:
  • Release Readiness Narratives
  • Failure Clustering And Trend Analysis
  • "What Changed?" Insights (Commit/Pr Correlation)
• Produce Outputs Consumable By Engineering Leadership And Teams (Dashboards, Markdown Summaries In Prs, Artifacts In Ci).

4) "Quality Gates" Via Agents

• Build Automated Review Agents That Evaluate User Stories/Requirements For Minimum Required Clarity And Data Before Development/Testing Starts:
  • Required Fields Present (Acceptance Criteria, Testable Outcomes, Data Needs, Dependencies)
  • Ambiguity Detection And Missing Edge Cases
  • Data/Privacy Considerations And Environment Needs
• Integrate Gates Into Workflow (Pr Checks, Issue Templates, Github Actions) To Reduce Churn And Rework.

Required Technical Skills (Must-Have)

Genai / Llm Agentic Development

• Hands-On Experience Building Llm-Powered Agents (Tool-Using, Multi-Step Reasoning, Guardrails).
• Experience With Prompting Patterns, Structured Outputs (Json Schemas), Evaluation, And Reducing Hallucinations.
• Ability To Design Agent Workflows For:
  • Test Generation/Augmentation
  • Requirements Review And Completeness Validation
  • Report Generation And Summarization