What are the responsibilities and job description for the GEN AI Engineer position at Jade Business Services (JBS)?
Fulltime Role
Job Title: Gen AI Engineer
Location: Houston (Hybrid)
Job Summary
We are seeking a highly skilled Gen AI Engineer to design, develop, and optimize AI evaluation and automation pipelines for enterprise-scale AI and agentic systems. The ideal candidate will have hands-on experience building evaluation frameworks for AI agents, supporting RAG pipelines, and automating weekly evaluation workflows using golden datasets and dynamic evaluation techniques.
- This role requires strong expertise in AI/LLM evaluation methodologies, Databricks environments, and automation frameworks. Experience with agentic workflows, LangGraph, and RAGAS is highly preferred.
Key Responsibilities
- Design and implement AI evaluation pipelines for LLM-based applications and AI agents.
- Develop automated evaluation workflows using golden datasets, benchmark testing, and dynamic evaluation methods.
- Build and maintain weekly evaluation jobs for AI agents and generative AI systems.
- Support and optimize RAG (Retrieval-Augmented Generation) pipelines for enterprise AI applications.
- Create scalable automation pipelines within Databricks environments.
- Develop evaluation metrics and monitoring strategies for AI model performance, accuracy, hallucination detection, and response quality.
- Collaborate with cross-functional teams including Data Science, ML Engineering, and Product teams.
- Assist in developing and improving agentic workflows and AI agent orchestration systems.
- Troubleshoot and optimize AI pipeline performance and workflow reliability.
- Document architecture, evaluation methodologies, and operational procedures.
Required Skills
- Strong experience in AI/LLM evaluation pipelines.
- Hands-on experience with AI evaluation frameworks and benchmarking methodologies.
- Experience developing automation workflows and scheduled pipeline jobs.
- Strong experience working in Databricks environments.
- Proficiency in Python and AI/ML ecosystem tools.
- Experience with REST APIs, workflow orchestration, and data processing pipelines.
- Understanding of prompt evaluation, hallucination detection, and model performance analysis.
Preferred / Nice-to-Have Skills
- Experience with agentic workflows and AI agent development.
- Hands-on experience creating AI agents and orchestration systems.
- Experience with LangGraph.
- Experience with RAGAS and RAG evaluation frameworks.
- Experience supporting and optimizing RAG pipelines.
- ADP platform experience is a plus.
- Technical Environment
- Python
- Databricks
- LangGraph
- RAGAS
- LLM Evaluation Frameworks
- AI/ML Pipelines
- Agentic AI Workflows
- RAG Architectures
- Automation & Scheduling Tools
Qualifications
- Bachelor’s or Master’s degree in Computer Science, Data Science, AI, or related field.
- Proven experience in Generative AI, Machine Learning, or AI Engineering roles.
- Strong analytical and problem-solving skills.
- Excellent communication and collaboration abilities.