What are the responsibilities and job description for the Lead Data Engineer position at Headway Tek Inc?
Lead Data Engineer – AWS (Data Lake, Redshift, Aurora Integration)
Location: [Chicago, IL] Hybrid
Duration : 1 Year
Key Responsibilities
Data Engineering & Architecture
• Design, build, and optimize data pipelines using AWS Glue, Lambda, and Step Functions for ingestion, transformation, and curation.
• Develop and maintain data lakes (S3 Lake Formation) and data warehouses (Redshift) to support analytics and visualization.
• Integrate Aurora (PostgreSQL/MySQL) as a serving layer for APIs and dashboards.
• Design APIs or direct query interfaces that expose curated datasets for React-based dashboards and web applications.
• Define and implement data models, partitioning strategies, and schema evolution best practices for performance and scalability.
Security, Governance & Compliance
• Implement data security policies, encryption, and role-based access controls using AWS IAM, KMS, Lake Formation, and Secrets Manager.
• Ensure compliance with organizational data governance and privacy standards.
• Maintain data cataloging, lineage, and access tracking within AWS Glue Data Catalog and Lake Formation.
• Implement CI/CD pipelines for data workflows in collaboration with DevOps teams.
________________________________________
Required Skills & Qualifications
• 8 years of experience in data engineering and AWS data architecture.
• Strong expertise with AWS Glue, Redshift, S3, Lake Formation, Aurora (Postgres/MySQL).
• Proven experience building APIs or exposing curated datasets for web applications (preferably React or similar front-end frameworks).
• Strong programming skills in Python and SQL.
• Experience with data modeling, ETL/ELT frameworks, and performance optimization.
• Working knowledge of Terraform/CloudFormation and CI/CD practices.
• Understanding of data security, IAM policies, encryption, and governance frameworks.
________________________________________
Preferred Skills
• Experience with Athena for ad-hoc analytics.
• Familiarity with API Gateway, AppSync, or GraphQL APIs for serving data to front-end applications.
• Knowledge of event-driven pipelines (Kinesis, EventBridge) and data quality monitoring.