AWS Data Engineer

Job Title: AWS Data Engineer

Location: Remote

Employment Type: Contract / Full-Time



Job Summary

We are seeking a highly skilled AWS Data Engineer with strong experience in building scalable cloud-native data platforms and ETL pipelines on AWS. The ideal candidate should have hands-on expertise with AWS Glue, Spark ETL, Step Functions, Amazon EKS, Kubernetes, Lambda, S3 Data Lake architectures, SQS, KEDA, Glue Data Catalog, Glue Data Quality, and CloudWatch.



Key Responsibilities

Design, develop, and maintain scalable ETL pipelines using AWS Glue and Apache Spark (PySpark).

Build and orchestrate data workflows using AWS Step Functions.

Design and implement S3 Data Lake architectures following AWS best practices.

Develop and deploy containerized applications on Amazon EKS using Kubernetes.

Build event-driven data processing solutions using Amazon SQS, AWS Lambda, and KEDA for auto-scaling.

Manage metadata using AWS Glue Data Catalog.

Implement data validation and governance using AWS Glue Data Quality.

Monitor applications and data pipelines using Amazon CloudWatch.

Optimize ETL jobs, Spark workloads, and Kubernetes deployments for performance and scalability.

Collaborate with architects, developers, DevOps, and business stakeholders to deliver robust cloud data solutions.

Participate in Agile ceremonies including sprint planning, code reviews, and production deployments.

Required Skills

5 years of experience in Data Engineering or Cloud Engineering.

Strong hands-on experience with AWS Glue.

Experience building Spark ETL pipelines using PySpark.

Experience with AWS Step Functions.

Strong knowledge of Amazon S3 Data Lake architecture and storage patterns.

Hands-on experience with Amazon EKS and Kubernetes.

Experience implementing event-driven architectures using Amazon SQS, AWS Lambda, and KEDA.

Experience with AWS Glue Data Catalog and Glue Data Quality.

Strong experience monitoring AWS workloads using Amazon CloudWatch.

Good understanding of distributed data processing and cloud-native application design.

Experience working in Agile/Scrum environments.



Preferred Skills

Python / PySpark

SQL

Terraform or CloudFormation

Docker

Git

CI/CD (Jenkins, GitHub Actions, CodePipeline)

Kafka

Apache Airflow

Delta Lake / Iceberg / Lake Formation

IAM, VPC, CloudTrail



Education

Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.



Mandatory Skills

AWS Glue

Spark ETL / PySpark

AWS Step Functions

Amazon S3 Data Lake

Amazon EKS

Kubernetes

Amazon SQS

KEDA

AWS Lambda

Glue Data Catalog

Glue Data Quality

Amazon CloudWatch



Nice to Have

Terraform

Docker

Python

SQL

CI/CD

Kafka

AWS Certifications (Solutions Architect, Developer, Data Engineer, or DevOps Engineer)