What are the responsibilities and job description for the Data Engineer (AWS Glue, Airflow, Lambda, Postgres) position at Cynet Systems?
We are looking for a Data Engineer (AWS Glue, Airflow, Lambda, Postgres) for our client in Fort Mill, SC
Job Title: Data Engineer (AWS Glue, Airflow, Lambda, Postgres)
Job Location: Fort Mill, SC
Job Type: Contract
Job Overview:
Pay Range: $60hr - $65hr
Responsibilities:
- Design and develop scalable data pipelines using AWS Glue, Lambda, and Event Bridge for event-driven and batch processing.
- Build and maintain AWS Glue ETL jobs (Spark/Python) for data ingestion, transformation, and curation across data lake layers.
- Develop and manage Airflow DAGs (MWAA or self-managed) to orchestrate data workflows, triggers, and dependencies across AWS services.
- Implement event-driven architectures using AWS Lambda and Event Bridge for near real-time data processing.
- Write, optimize, and maintain complex SQL queries in Postgres for data validation, transformation, and reporting.
- Manage metadata and schema definitions using AWS Glue Data Catalog, ensuring proper governance and discoverability.
- Build and support robust data ingestion frameworks for batch and near real-time data pipelines.
- Monitor and troubleshoot pipeline performance using AWS monitoring tools (CloudWatch, logs, alerts).
- Collaborate with downstream teams (BI, analytics, and Snowflake if applicable) to ensure reliable data consumption.
- Contribute to system reliability, scalability, and performance optimization of the data platform.
- 6 years of experience in Data Engineering or a related field.
- Strong hands-on expertise in AWS Glue, Lambda, Event Bridge, and S3.
- Experience building and orchestrating workflows using Apache Airflow (MWAA preferred).
- Strong proficiency in SQL (Postgres preferred) for complex transformations and performance tuning.
- Experience with Python / Spark for ETL development.
- Solid understanding of event-driven data architecture and pipeline design.
- Familiarity with AWS IAM, CloudWatch, and logging/monitoring frameworks.
- Experience with CI/CD pipelines (e.g., GitHub Actions) for deployment automation.
- Experience with Terraform or infrastructure-as-code tools.
- Knowledge of data lake formats (Parquet, Iceberg, Delta).
- Understanding of data modeling, partitioning, and schema evolution.
- Familiarity with AWS services such as Lake Formation, SNS, and CloudTrail.
- Exposure to Snowflake or other data warehouse platforms is a plus.
Salary : $60 - $65