What are the responsibilities and job description for the Senior Data Engineer position at Anagh Technologies Inc?
Job Title: Senior Data Engineer (Level 4)
Location: Malvern, PA
Job Descripti
onWe are seeking a highly skilled Senior Data Engineer (Level 4) to design, develop, and optimize enterprise-scale data pipelines supporting Vanguard's B2B Service Experience and Financials initiatives. The ideal candidate will have strong expertise in Python, Apache Spark (PySpark), AWS Glue, and AWS cloud services, with experience building scalable ETL/ELT solutions and modern Data Lake architecture
s.Required Skil
- ls7 years of Data Engineering experience in enterprise environment
- s.Strong proficiency in Python and advanced SQ
- L.Extensive experience with Apache Spark (PySpark
- ).Hands-on experience with AWS Glue, S3, Athena, Lambda, Step Functions, and EventBridg
- e.Experience building scalable ETL/ELT pipeline
- s.Strong understanding of Data Lake concepts and Medallion Architectur
- e.Experience with CloudFormation or Terrafor
- m.Hands-on experience implementing CI/CD pipelines using GitHub Action
- s.Knowledge of distributed data processing and dimensional data modelin
- g.Experience working with Git, Agile/Scrum methodologies, and production suppor
- t.Excellent communication and stakeholder management skill
s.Preferred Skil
- lsFinancial Services or Asset Management domain experienc
- e.Experience with Snowflake or Databrick
- s.Knowledge of Apache Iceberg, Delta Lake, or Hud
- i.Experience with Apache Airflo
- w.Familiarity with Kafka or Amazon Kinesi
- s.Experience supporting AI/ML and advanced analytics platform
- s.AWS Professional or Specialty Certificatio
n.Responsibiliti
- esDesign, develop, and maintain high-performance data pipelines using Python, PySpark, and AWS Glu
- e.Build and optimize scalable ETL/ELT workflows for enterprise data platform
- s.Orchestrate data ingestion from legacy and modern data source
- s.Develop and maintain Data Lake solutions following Medallion Architectur
- e.Ensure data quality, reliability, and performance across production environment
- s.Implement Infrastructure as Code using CloudFormation/Terrafor
- m.Build and maintain CI/CD pipelines using GitHub Action
- s.Collaborate with business analysts, architects, and cross-functional teams to deliver data solution
- s.Troubleshoot production issues and optimize distributed data processing workload
- s.Follow Agile best practices and contribute to continuous improvement initiative