What are the responsibilities and job description for the GenAI Data Engineer position at Spar Information Systems?
Role: Gen AI Data Engineer
Location: Charlotte, NC (Hybrid from Day 1)
Duration: 10 Months
Job description:
Key Responsibilities:
GenAI Platform & Utility Development
- Design and develop generic, reusable utilities for GenAI data ingestion, context orchestration, and retrieval pipelines.
- Build standardized data movement patterns to move structured and unstructured data between systems, vector stores, and LLMs.
- Implement MCP-based tooling and services to enable consistent interaction between LLMs, tools, and enterprise data.
- Develop config-driven and metadata-driven frameworks to support multiple GenAI use cases. RAG Architecture & Implementation
- Design and implement RAG pipelines including:
- Data ingestion and chunking utilities o Embedding generation and lifecycle management
- o Retrieval strategies (semantic, hybrid, metadata-based)
- Build shared components for prompt construction, context enrichment, and grounding.
- Ensure low-latency, high-quality retrieval across large-scale datasets. API & Integration Development
- Build and expose robust APIs for GenAI services, utilities, and data movement workflows.
- Enable integration with upstream and downstream systems through REST and event-driven interfaces.
- Implement secure authentication, authorization, and rate limiting for AI services. Engineering Excellence & Governance
- Write production-grade, well-tested, and maintainable code.
- Implement observability (logging, tracing, metrics) for GenAI utilities.
- Address key GenAI concerns including:
- Security and access control
- Cost management and optimization
- Prompt/version lifecycle management
- Document utilities, architectures, and usage patterns for widespread adoption.
- Collaborate with platform, data, and application teams to define GenAI standards and best practices.
Required Skills & Qualifications:
Core GenAI Skills:
- Hands-on experience with Generative AI / LLM-based systems in production.
- Strong experience with Model Context Protocol (MCP) for tool and data orchestration.
- Proven experience designing and implementing RAG architectures.
- Expertise in prompt design, context management, and retrieval strategies. Data Movement & Platform Skills
- Experience building generic utilities for data ingestion, transformation, and retrieval.
- Strong understanding of unstructured data handling (documents, logs, text, metadata).
- Familiarity with vector databases and embedding workflows.
- Experience designing scalable, reusable platform-level components. API & Software Engineering
- Strong experience designing and implementing RESTful APIs.
- Proficiency in one or more backend languages (Java, Python, or similar).
- Experience with asynchronous processing and event-driven systems.
- Strong understanding of distributed systems and scalability patterns.
Preferred Qualifications:
- Experience building enterprise GenAI platforms or internal tooling.
- Familiarity with agent-based architectures and AI workflows.
- Experience integrating GenAI systems with data platforms and enterprise systems.
- Knowledge of cloud-native architectures (AWS, Azure, or Google Cloud Platform).
- Experience with containerization and CI/CD pipelines.
- Strong focus on responsible AI, governance, and compliance.