SLG - 26 - 12551 - AI Testing Engineer - REMOTE - SLG - DEL - 79876

Greetings from Sligo Software Solutions, Inc. !!

SLIGO is a Software Development & IT Consulting company. We have around 100 IT associates working with our esteemed clients across several domains such as IT, Banking, Healthcare, and Government.

We have an urgent requirement for AI Testing Engineer - REMOTE position with our client. Role: AI Testing Engineer - REMOTE Location: Albany, New York Duration: 1 year 1 day(s) Job ID: SLG - 26 - 12551 Client: NYS - DOH

As part of Client transformation, the program is developing AI and GenAI capabilities to improve how members and staff interact with the system, streamline manual processes, and make enterprise knowledge easier to access.

Role Overview

• Support testing and quality assurance of AI/GenAI solutions, including conversational AI, RAG, Agentic AI.
• Validate AI solutions meet functional, business, and technical requirements.
  
  

Preferred Qualifications

• Experience with software QA/testing; AI/ML or GenAI testing experience.
• preferred Familiarity with LLMs and AI evaluation techniques.
• Experience with test management / defect tracking tools (e.g., Jira).
• Strong analytical skills and ability to identify unexpected AI behavior.
• Experience working in regulated or public-sector environments in a plus.
  
  

Key Responsibilities

• Develop and execute test plans, test cases, and evaluation scenarios for AI solutions.
• Test AI outputs for accuracy, relevance, consistency, completeness, etc.
• Perform negative, edge-case, adversarial testing.
• Validate content retention, guardrails, quality, etc.
• Track testing metrics and provide clear readiness recommendations