What are the responsibilities and job description for the Data Engineer - Full Time - Malvern, PA position at INUIX Consulting?
Hello All,
Job Description
Must Have Technical/Functional Skills
Python, Glue ETL, Attunity, Pyspark, AWS (S3, IAM, Lambda, Cloud Formation)
Roles & Responsibilities
Design, develop, and maintain scalable ETL/data integration pipelines using AWS Glue, PySpark, and Python.
Develop and optimize data transformation workflows to process large datasets from multiple source systems.
Utilize Attunity for real-time and batch data replication between on-premises and cloud environments.
Build and manage data lakes and storage solutions using Amazon S3, ensuring data quality, security, and governance.
Implement serverless data processing solutions using AWS Lambda and integrate with other AWS services.
Configure and manage IAM roles, policies, and permissions to ensure secure access to AWS resources.
Automate infrastructure provisioning and deployment using AWS CloudFormation.
Monitor, troubleshoot, and optimize ETL jobs, data pipelines, and cloud resources for performance and reliability.
Collaborate with business, analytics, and application teams to gather requirements and deliver data solutions.
Support CI/CD processes, perform code reviews, and maintain technical documentation following best practices.
Salary : $120,000 - $140,000