AI Architect

Role: AI Architect
Location: Jersey City, NJ (Onsite)(Locals Only)
Exp: 12 Years

Job Description

12 years in software engineering, and AI architecture, with 3 years designing and delivering GenAI/LLM solutions and at least two enterprise AI implementations taken to production.

AI Architect to define how enterprise AI gets built — and built to last. You will own the target-state architecture for client AI programs, set the platform, model, and governance direction, and guide engineering teams from discovery and PoC through to secure, scalable, production-grade agentic solutions.
The role sits at the intersection of AI platform architecture, solution design, and business value — best suited to a seasoned technologist who can make sound architectural trade-offs under ambiguity, still writes code when it matters, and is the trusted technical authority for client CXOs, enterprise architects, and delivery teams.
Roles & Responsibilities

AI and agentic architecture
·       Deep expertise in agentic system design — single and multi-agent orchestration, planning and reasoning strategies, tool/function calling, MCP, memory, guardrails, and human-in-the-loop patterns.
·       Strong command of RAG and context engineering at scale: chunking and indexing strategy, embeddings, vector databases, hybrid and re-ranked retrieval, knowledge graphs, and retrieval evaluation.
·       Hands-on with LangChain/LangGraph, LlamaIndex, CrewAI, AutoGen/Semantic Kernel or equivalent frameworks, plus OpenAI/Anthropic APIs, Hugging Face, and open-weight models.

Platform and engineering
·       Strong Python engineering and API design (FastAPI or similar), microservices and event-driven integration patterns, and enterprise system integration.
·       Production experience with Docker, Kubernetes (EKS/AKS/GKE), Terraform or equivalent IaC, CI/CD, and Git-based engineering practices.
Proven LLMOps/AgentOps experience: evaluation frameworks, observability and tracing (e.g. LangSmith, Langfuse, Arize, MLflow), prompt/model lifecycle, and cost and latency optimisation.
Architecture-level depth across frontier-model providers and hyperscaler AI platforms, with proven production depth in at least one
Able to justify platform and model choices against client constraints — capability, context window, latency, cost per interaction, data residency, enterprise controls, and existing cloud commitments.

Leadership
·       Proven ability to own architecture decisions, document them clearly (ADRs, reference architectures, HLD/LLD), and defend them with business and technical stakeholders.
·       Excellent communication and executive presence; able to hold a CXO room, an enterprise architecture board, and an engineering stand-up with equal credibility.
·       Track record of leading and mentoring cross-functional AI and engineering teams across multiple concurrent engagements.

Good to have
·       Experience with modern data platforms and their GenAI surfaces (Databricks Mosaic AI, Snowflake Cortex, Lakehouse patterns) and data governance tools.
·       Certification as a cloud solutions architect or on a hyperscaler AI/ML track (AWS, Azure, or Google).
·       Pre-sales, consulting, or forward-deployed engineering experience, including proposal and SoW ownership.
·       Self-hosted / open-weight model deployment and optimisation (vLLM, Ollama, TGI, quantisation, GPU sizing) for strict data-boundary requirements.
Familiarity with AI regulation and standards such as the EU AI Act, NIST AI RMF, and ISO/IEC 42001.