What are the responsibilities and job description for the Lead Data Engineer (Snowflake, Airflow and Graph RAG) - W2 Role position at Saransh Inc?
Role: Lead Data Engineer (Snowflake, Airflow and Graph RAG)
Location: New York, NY / Westport, CT (Onsite from Day 1)
Job Type: W2 Contract
Project Duration: 12 months
Requirements:
Technical Expertise:
- Strong proficiency in Snowflake and Airflow (mandatory).
- Deep knowledge of GraphRAG and Graph Data Structures.
- Strong background in Data Architecture/Design with experience in market data sources.
AI and Integration Experience:
- Experience integrating heavy AI-driven architectures.
- Ability to share knowledge with project and client teams on Cypher, Graph, and RAG traversals.
Additional Skills:
- AWS experience is a plus.
Responsibilities:
Data Integration and Management:
- Identify and integrate structured, unstructured, and semi-structured data sources.
- Implement RBAC and develop logical data models, ensuring compliance and data lineage.
- Document business transformations and establish data quality rules.
- Create a comprehensive data catalog and register data assets with metadata.
Design Implementation:
- Translate and implement the architectural blueprint for the Data Factory.
- Ensure robust data management and quality to support automated investment processes and AI-driven analytics.
- Implement Neo4j-powered Knowledge Graph and ensure investment decision traceability.
Advanced Data Solutions:
- Develop and maintain multi-dimensional asset ontology and temporal versioning.
- Incorporate GraphRAG for transforming unstructured documents into queryable graph entities.
Security and Access Control:
- Implement property-based access control and oversee real-time data ingestion pipelines.
Data Architecture and Management:
- Oversee scalable data warehousing solutions using Snowflake.
- Develop frameworks for data quality and governance, including metadata registration and sensitivity/security considerations.
Note: Visa Independent candidates are preferred
