What are the responsibilities and job description for the Lead Data Engineer position at Intellectt Inc?
Number of Positions: 1 Lead and 1 Senior (Both Required).
Interview Mode: L1 & In-person in NJ.
Key Skills Required:
- AWS (S3, Redshift, Glue, Lambda, EMR, Athena).
- Data Engineering & Data Modeling (Star Schema, Snowflake, Dimensional Modeling).
- Python, PySpark, SQL.
- Big Data Technologies (Hadoop, Spark).
- Infrastructure as Code (Terraform).
- AI/ML integration basics.
- Visualization tools (Power BI).
Roles & Responsibilities:
- Design, develop, and maintain scalable data pipelines for batch and real-time processing using AWS services
- Build and optimize data lakes and data warehouses using Amazon S3, Redshift, and Glue
- Develop robust ETL/ELT pipelines using Python, PySpark, and SQL
- Implement efficient data modeling techniques such as star schema and dimensional modeling
- Work with large-scale distributed systems using Hadoop and Apache Spark
- Integrate AI/ML models into data pipelines to support advanced analytics
- Automate infrastructure provisioning using Terraform (IaC)
- Ensure data quality, governance, and security across pipelines
- Collaborate with cross-functional teams including data scientists, analysts, and business stakeholders
- Develop dashboards and reports using Power BI for business insights
- Monitor and optimize performance of data pipelines and cloud resources.
- Exposure to AI/ML frameworks (SageMaker, TensorFlow, etc.)
Salary : $48 - $56