What are the responsibilities and job description for the Senior AI Data Annotation Jobs in Seattle position at Rex.zone?
Role Summary
Rex.zone is hiring for a remote, full-time Senior AI Data Annotation role supporting training-data quality for modern AI/ML systems. You will deliver high-precision human feedback across RLHF, LLM evaluation, prompt evaluation, and QA evaluation to drive measurable model performance improvement.
Key Responsibilities
Apply via Rex.zone and highlight relevant AI data annotation, RLHF, LLM evaluation, prompt evaluation, and guideline-driven QA evaluation experience.
Rex.zone is hiring for a remote, full-time Senior AI Data Annotation role supporting training-data quality for modern AI/ML systems. You will deliver high-precision human feedback across RLHF, LLM evaluation, prompt evaluation, and QA evaluation to drive measurable model performance improvement.
Key Responsibilities
- Deliver senior-level AI data annotation across text, image, and mixed-modality tasks
- Execute RLHF workflows including preference ranking, pairwise comparisons, and (when required) rationale capture
- Perform rubric-based LLM evaluation and QA evaluation for helpfulness, honesty, harmlessness, and policy compliance
- Run prompt evaluation to identify failure modes, jailbreak risk, and instruction-following gaps
- Apply strict annotation guidelines compliance; report ambiguities and propose guideline improvements
- Conduct error analysis, label auditing, calibration, and consensus reviews to stabilize annotation quality
- Support content safety labeling (toxicity, self-harm, violence, hate, regulated content) with structured QA review
- Collaborate with engineers/data ops on task design, schema updates, and quality monitoring signals
- Document edge cases clearly and escalate issues that impact training data quality
- Mid-Senior experience in AI data annotation, data labeling, and/or production LLM evaluation
- Understanding of RLHF concepts and how human feedback improves model alignment
- Proven ability to follow and improve annotation guidelines with high attention to detail
- Experience with QA evaluation methods (audits, gold sets, calibration, inter-annotator agreement)
- Familiarity with NLP labeling (classification, sentiment, NER) and/or computer vision annotation
- Excellent written communication for documenting decisions, rationales, and edge cases
- Ability to work remotely within the US and handle sensitive data securely
- Hourly pay range: $30–$50 per hour (USD)
Apply via Rex.zone and highlight relevant AI data annotation, RLHF, LLM evaluation, prompt evaluation, and guideline-driven QA evaluation experience.
Salary : $30 - $50