What are the responsibilities and job description for the AI Observability Engineer position at Holistic Partners, Inc?
Job Title: AI Observability Engineer
Location: Charlotte, NC / Philadelphia, PA (Hybrid)
Duration: 6 month
Interview Process: Video
Visa Requirements: U.S. Citizens/Green Card ONLY due to legal or government contract requirement
Tax Term: W2 Only
Responsibilities:
We are seeking a highly skilled Senior AI/ML Engineer with expertise in Generative AI, Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), Agentic AI, and Machine Learning to design, develop, and deploy enterprise-grade AI solutions. The ideal candidate will have hands-on experience building production-ready AI applications using modern LLM frameworks, optimizing model inference, implementing AI observability, and deploying scalable AI services on cloud platforms.
Key Responsibilities:
- Design, develop, and deploy enterprise-grade Generative AI applications using OpenAI, AWS Bedrock, and Hugging Face models.
- Build scalable RAG (Retrieval-Augmented Generation) solutions using vector embeddings, Azure AI Search, LangChain, and LangGraph.
- Develop Agentic AI workflows utilizing multi-agent orchestration for intelligent task automation and decision-making.
- Fine-tune and optimize LLMs using LoRA, QLoRA, vLLM, PagedAttention, and continuous batching to improve inference performance and reduce latency.
- Design and implement REST APIs and AI microservices using Python and FastAPI.
- Develop scalable ML pipelines for model training, deployment, monitoring, and lifecycle management.
- Implement AI observability using tools such as Arize to monitor prompt quality, model performance, inference metrics (TTFT, TPOT), hallucinations, and response safety.
- Establish AI governance, guardrails, and evaluation frameworks to ensure responsible AI deployment.
- Develop machine learning models using PyTorch, Scikit-learn, XGBoost, and other ML frameworks.
- Collaborate with cross-functional teams including product managers, data engineers, and business stakeholders to deliver AI-powered business solutions.
- Optimize asynchronous processing pipelines to improve scalability and throughput.
- Build analytics dashboards and present AI-driven insights to business stakeholders.
- Deploy containerized AI services using Docker, GitHub, and CI/CD pipelines.