What are the responsibilities and job description for the LLM / GenAI Engineer position at Evlo AI?
About The Role
The role builds the LLM-powered systems at the core of the product: retrieval pipelines, agentic workflows, and evaluation infrastructure that determine whether AI features actually work in production.
This is a hands-on engineering position on a small, senior team — expect to ship code daily, own systems end-to-end, and make architecture decisions with real production consequences.
Key Responsibilities
The role builds the LLM-powered systems at the core of the product: retrieval pipelines, agentic workflows, and evaluation infrastructure that determine whether AI features actually work in production.
This is a hands-on engineering position on a small, senior team — expect to ship code daily, own systems end-to-end, and make architecture decisions with real production consequences.
Key Responsibilities
- Design and implement RAG pipelines end-to-end: chunking strategies, hybrid retrieval, reranking, and context optimization using LangChain, LlamaIndex, or custom architectures
- Build and scale vector database integrations (Pinecone, Weaviate, pgvector) handling millions of documents with sub-second retrieval latency
- Develop agentic workflows using tool-calling, function execution, and multi-step reasoning frameworks — deployed and monitored in production
- Own LLM evaluation infrastructure: golden datasets, LLM-as-judge pipelines, regression test suites, and automated quality gates in CI/CD
- Run parameter-efficient fine-tuning (LoRA, QLoRA) and structured output optimization on domain-specific datasets using PyTorch and Hugging Face
- Integrate LLM applications with backend services via FastAPI, streaming responses, and robust async patterns
- Instrument observability for GenAI systems: tracing (LangSmith/Langfuse), token cost tracking, latency dashboards, and hallucination detection alerts
- 3–7 years of software engineering experience, including at least 2 years building LLM/GenAI systems in production
- Deep proficiency in Python with experience in async programming, API design, and production-grade software practices
- Hands-on experience with at least one vector database and embedding model stack in a real deployment
- Practical understanding of LLM internals: tokenization, context windows, prompt engineering tradeoffs, and inference optimization (vLLM, quantization)
- Experience deploying GenAI features on AWS, GCP, or Azure, including Docker-based containerization and CI/CD pipelines
- Bachelor's degree in Computer Science, Engineering, or equivalent practical experience
- Bonus: experience fine-tuning open-source models (Llama, Mistral, Qwen), prior work on agentic systems, contributions to open-source LLM tooling, or a relevant ML publication