Principal AI Solutions Evaluation Lead

Description

Title: Principal AI Solutions Evaluation Lead

Location: Charlotte, NC

Duration: 18 months

Work Engagement: W2

Work Schedule: Hybrid 3 days in office/2 days remote

Benefits on offer for this contract position: Health Insurance, Life insurance, 401K and Voluntary Benefits

Summary:

In this contingent resource assignment, you may: Consult on complex initiatives with broad impact and large-scale planning for Specialty Software Engineering. Review and analyze complex multi-faceted, larger scale, or longer-term Specialty Software Engineering challenges that require in-depth evaluation of multiple factors, including intangibles or unprecedented factors. Contribute to the resolution of complex and multi-faceted situations requiring solid understanding of the function, policies, procedures, and compliance requirements that meet deliverables. Strategically collaborate and consult with client personnel. Required Qualifications: Specialty Software Engineering experience, or equivalent demonstrated through one or a combination of the following: work or consulting experience, training, military experience, education.

Key Responsibilities:

Key Requirements:

• Applicants must be authorized to work for ANY employer in the U.S. This position is not eligible for visa sponsorship.

• Deep expertise in designing and scaling reusable evaluation frameworks, including advanced test pattern design, comparative benchmarking models, and mature scoring methodologies that enable cross-solution standardization.
  • Advanced application of AI/ML and agentic systems to transform scouting, assessment, and evaluation lifecycles—enabling intelligent orchestration, automated competitive analysis, adaptive test generation, and production of investment-grade insights (e.g., structured memos, comparative scoring, and executive-ready synthesis)
  • Advanced Infrastructure-as-Code and automation capabilities, including modular architecture, policy-as-code, and CI/CD or GitOps patterns for complex, multi-environment evaluation setups.
  • Experience embedding automated evaluation ecosystems at scale, including extensible pipelines, reusable test harnesses, and integration with enterprise platforms for continuous evaluation.
  • Strong experience evaluating SaaS and emerging vendor solutions in regulated enterprise environments, with depth in integration complexity, identity/auth patterns, data flows, operational readiness, and long-term supportability.
  • Familiarity with enterprise governance and control frameworks, including third-party risk lifecycles, security review processes, and audit-quality evidence expectations in large regulated organizations.
  • Strong background in platform and infrastructure engineering (e.g. OpenShift, Observability), enabling deep technical judgment and architectural tradeoff analysis during evaluations.
  • Demonstrated ability to produce executive-level narratives and decision artifacts, including PoVs and “whitepaper-light” analyses that clearly articulate tradeoffs, risks, and recommendation rationale.
  • Experience in technology scouting and evaluation of emerging products, including identifying, screening, and assessing vendor capabilities against strategic business and technical needs.
  • Proven experience designing and delivering hackathons at scale, including structuring problem statements, evaluation criteria, judging frameworks, and end-to-end execution.
  • Advanced AI/ML certifications or equivalent applied expertise preferred.

• • Software engineering experience, including hands-on delivery of at
  least one production-grade Agentic AI solution (preferred).
  • Experience leading end-to-end PoC evaluations in collaboration with third-party lab/platform teams and vendors, including defining scope, success criteria, standardized test patterns, execution plans, and scoring frameworks, while coordinating environment readiness, connectivity, access, and execution workflows across diverse technologies.
  • Experience in cloud-native, containerized environments (public/private cloud), with the ability to assess integration, operability, scalability, and risk of vendor solutions.
  • Experience enabling repeatable environment configuration and building or supporting automated evaluation frameworks, including test orchestration, validation, and scoring, to standardize, scale, and accelerate PoC assessments while validating vendor deployment patterns.
  • Strong Agile execution skills in a fast-moving, cross-functional environment.
  • Excellent technical writing and stakeholder communication skills to produce decision-ready artifacts (PoV, findings summaries, scoring inputs, recommendations).