What are the responsibilities and job description for the Senior Data Engineer GenAI & Agentic AI With Financial Domain in NC position at Smart Caliber Technology?
Location: Charlotte, NC (Onsite)
Job Description
We are seeking a highly skilled and innovative Senior Data Engineer to join our growing Data & AI team. In this role, you will be responsible for building scalable data infrastructure, ensuring data quality and governance, and driving the integration of advanced AI and agentic systems into our enterprise data platform.
Key Responsibilities
Data Engineering
- Design, develop, and maintain scalable data pipelines for both batch and real-time data processing.
- Build robust data models, schemas, and transformations to support analytics, reporting, and AI workloads.
- Ensure data quality, reliability, lineage, observability, and governance across the data ecosystem.
- Optimize data storage, processing, and retrieval mechanisms to support high-performance applications.
- Collaborate with cross-functional teams to deliver scalable and reliable data solutions.
Agentic AI Engineering
- Design, develop, and support Agentic AI systems, including autonomous agents capable of reasoning, planning, and executing tasks.
- Implement tool integration, memory management, feedback loops, and self-refinement mechanisms for AI agents.
- Architect and manage multi-agent workflows, orchestration frameworks, and task decomposition strategies.
- Integrate Large Language Models (LLMs) into data pipelines for intelligent reasoning, enrichment, and automation.
- Partner with Data Science, Machine Learning, and AI teams to productionize agent-driven solutions.
- Develop and support AI architectures leveraging Retrieval-Augmented Generation (RAG), vector embeddings, prompt engineering, and related AI patterns.
Required Qualifications
- 4 years of experience in Data Engineering, Data Platforms, or a related field.
- Strong proficiency in SQL and Python.
- Hands-on experience with distributed data processing frameworks such as Apache Spark and/or Databricks.
- Experience working with cloud platforms such as AWS, Azure, or Google Cloud Platform (Google Cloud Platform).
- Familiarity with Large Language Models (LLMs), vector databases, AI/ML pipelines, and modern AI ecosystems.
- Strong understanding of data modeling, data quality, observability, and governance best practices.
- Excellent analytical, problem-solving, and communication skills.
Preferred Qualifications
- Experience building Agentic AI or Multi-Agent AI systems.
- Knowledge of RAG architectures, vector embeddings, prompt engineering, and LLM application development.
- Experience with AI orchestration frameworks such as LangChain, LlamaIndex, CrewAI, AutoGen, or similar technologies.
- Exposure to MLOps, AI deployment pipelines, and production AI systems.
Key Skills
Data Engineering: SQL, Python, Spark, Databricks, Data Modeling, ETL/ELT, Data Quality, Data Observability, Cloud Platforms (AWS/Azure/Google Cloud Platform)
AI & GenAI: LLMs, Agentic AI, Multi-Agent Systems, RAG, Vector Databases, Embeddings, Prompt Engineering, LangChain, LlamaIndex, AI/ML Pipelines
Cloud & Platform: AWS, Azure, Google Cloud Platform, Data Lake, Data Warehouse, Real-Time Processing, Scalable Data Architecture