What are the responsibilities and job description for the Senior Data Annotation (Austin) position at Rex.zone?
Senior Data Annotation (Austin) — Remote (US)
Rex.zone is hiring for a full-time Senior Data Annotation role aligned to the Austin market, with day-to-day work performed remotely within the US. You will lead end-to-end data annotation and evaluation workflows that support AI/ML training pipelines, including RLHF, prompt-response evaluation, and rubric-based scoring for large language model evaluation.
What You Will Do
This is a Remote role for candidates located in the US, aligned to Austin hiring intent. You will work asynchronously with distributed teams and participate in calibration sessions to maintain training data quality and rubric consistency.
Compensation
Hourly base pay range: $30–$50 per hour, depending on experience and scope of QA ownership.
How To Apply
Apply via Rex.zone and include a short summary of your data annotation, RLHF, and QA evaluation experience, plus examples of guidelines you have followed or improved (no confidential data).
Rex.zone is hiring for a full-time Senior Data Annotation role aligned to the Austin market, with day-to-day work performed remotely within the US. You will lead end-to-end data annotation and evaluation workflows that support AI/ML training pipelines, including RLHF, prompt-response evaluation, and rubric-based scoring for large language model evaluation.
What You Will Do
- Execute and review data labeling for NLP and (as needed) computer vision datasets
- Perform RLHF evaluations (preference ranking, pairwise comparisons, structured scoring)
- Run QA evaluation audits for accuracy, consistency, and policy alignment
- Conduct prompt evaluation for helpfulness, correctness, and safety
- Apply named entity recognition tagging and taxonomy mapping where required
- Support content safety labeling using policy-based classification
- Write/refine annotation guidelines, edge-case notes, and decision logs
- Track dataset health metrics (agreement rate, defect density, rework rate) and propose fixes
- Collaborate with engineering on tooling feedback (labeling UI, workflow automation, sampling strategy)
- Experience in data annotation, data labeling, or QA evaluation for AI/ML systems
- Familiarity with RLHF and large language model evaluation methodologies
- Ability to follow and improve annotation guidelines compliance under ambiguity
- Strong attention to detail and consistent rubric-based judgment
- Experience with NLP tasks such as classification, summarization evaluation, and named entity recognition
- Professional communication for documenting edge cases and explaining labeling decisions
This is a Remote role for candidates located in the US, aligned to Austin hiring intent. You will work asynchronously with distributed teams and participate in calibration sessions to maintain training data quality and rubric consistency.
Compensation
Hourly base pay range: $30–$50 per hour, depending on experience and scope of QA ownership.
How To Apply
Apply via Rex.zone and include a short summary of your data annotation, RLHF, and QA evaluation experience, plus examples of guidelines you have followed or improved (no confidential data).
Salary : $30 - $50