What are the responsibilities and job description for the Medical AI Evaluator (Remote, Hourly Contractor) position at CNTXT AI?
Position Summary
In this remote, hourly contractor role, you will evaluate AI-generated medical content and develop cases that test clinical reasoning accuracy. Your work directly improves how leading AI models handle healthcare information, making them more accurate, reliable, and safe. Tasks may include:
Evaluating AI-generated medical responses for clinical accuracy, reasoning quality, and patient safety implications
Identifying errors in clinical methodology, unsafe assumptions, missing contraindications, and misinterpretation of diagnostic data
Writing clear, precise feedback explaining corrections and reasoning gaps
Developing prompts and test cases that probe AI accuracy across clinical scenarios
Rating and comparing AI responses based on correctness, internal consistency, and contextual appropriateness
Fact-checking medical content against reliable sources using consistent reasoning
Profile Requirements:
Bachelor's degree or higher in Medicine (MD/DO), Nursing, Public Health, Health Sciences, or Allied Health
5 years of professional experience in a relevant healthcare discipline
Strong clinical reasoning skills: differential diagnosis, risk stratification, red-flag recognition
Solid grounding in disease processes, patient care, public health principles, and medical terminology
Full professional English proficiency
Exceptional attention to detail and ability to explain corrections clearly in writing
Reliable and self-directed, with consistent output quality in a remote, asynchronous workflow
Preferred Experience:
Experience in clinical documentation review, utilization review, or healthcare editorial QA
Prior experience with AI data training or annotation
About CNTXT AI
CNTXT AI builds artificial intelligence products and data solutions with a focus on making AI accurate, safe, and globally relevant for impact. Our work spans data services, custom AI solutions, and proprietary AI products, with deep expertise in Arabic-native and secure, sovereign solutions.