What are the responsibilities and job description for the AI Developer position at GuruSchools LLC?
AI Developer Position Responsibilities Key Responsibilities
- Design, implement, and operate Retrieval-Augmented Generation (RAG) services using Azure AI/Search, including chunking, embeddings, re-ranking, evaluation, and citation display.
- Design and deploy Model Context Protocol (MCP) tools/servers to integrate security scanners, inventory systems, approvals, and Azure DevOps/GitHub services.
- Build agentic AI solutions using AutoGen, CrewAI, and/or Agno, enabling secure tool-calling and multi-agent orchestration for troubleshooting and workflow automation.
- Develop production-grade chatbots (multi-turn, retrieval-grounded) with prompt management, guardrails, audit logging, and telemetry.
- Integrate Azure OpenAI securely behind API Management (APIM), manage secrets with Key Vault, handle events via Event Hub, and instrument with App Insights/Log Analytics.
- Evaluate and (where appropriate) fine-tune open-source models (e.g., PEFT/LoRA), balancing quality, latency, cost, and safety.
- Ship with CI/CD on Azure DevOps, implement unit/integration tests, red-team for prompt-injection/jailbreaks, and document runbooks.
- 4-8 years total software development experience, with 2 years in applied LLM/GenAI.
- Strong Python skills and hands-on experience with Azure OpenAI and Azure AI/Search (vector search, hybrid search, semantic ranking).
- Practical experience with agent frameworks (AutoGen, CrewAI, Agno) and MCP/tool-use patterns.
- Proven Azure PaaS experience: Azure Functions or Web Apps, APIM, Key Vault, Event Hub; familiarity with Entra ID/RBAC and secure API design.
- Experience implementing observability (App Insights, Log Analytics/KQL) and CI/CD with Azure DevOps.
- RAG evaluation frameworks (e.g., Ragas), custom golden sets, KQL proficiency, Cosmos DB familiarity.
- Security-first mindset: content safety, prompt-injection defenses, data privacy controls, and threat modeling for AI systems.
- Experience with cost monitoring/optimization of LLM workloads and latency tuning.
- Certifications (any of):
- Microsoft Certified: Azure AI Engineer Associate (AI-102)
- Microsoft Certified: Azure Developer Associate (AZ-204)
- Microsoft Certified: Azure Solutions Architect Expert (AZ-305)
- Microsoft Certified: Azure Data Scientist Associate (DP-100)