What are the responsibilities and job description for the AWS Cloud Data Engineer position at ALTEN?
Job Overview:
- We are seeking an AWS-Data Engineer with strong experience across the entire Cloud Data stack. The ideal candidate should have extensive experience in data pipelines (ELT/ETL), data warehousing and dimensional modeling, and curation of data sets for Data Scientists and Business Intelligence users. Experience in building Data Lake. This candidate should also have excellent problem-solving ability dealing with large volumes of data and various data sources.
- Design, Build and Operate the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, cloud migration tools, and 'big data' technologies.
- Optimize various RDBMS engines in the cloud and solve customers security, performance, and operation problems.
- Design, Build, and Operate large, complex data lakes that meet functional / non-functional business requirements.
- Optimize various data types of ingestion, storage, processing, and retrieval from near real-time events, and IoT, to unstructured data as images, audio, video, and documents, and in between.
- Work with customers and internal stakeholders including the Executive, Product, Data, Software Development, and Design teams to assist with data-related technical issues and support their data infrastructure and business needs.
- Understand and Develop AI/ML models using AWS AI/ML services (e.g., Sage Maker, Rekognition, Comprehend) and collaborate with data scientists and engineers to implement machine learning solutions
- 5 years of experience in a Data Engineer role in a cloud-native eco-system.
- 5 years working experience in AWS Glue ETL, Redshift, Step Function, Athena
- Experience in implementing data pipelines for both streaming and batch integrations using tools/frameworks like Glue ETL, Lambda, Spark, Spark Experience in Lake Formation
- Orchestration, Maintain and enhance models for SDLC using CI/CD pipelines in AWS.
- Experience in Relational and NoSQL databases, such as MySQL or Postgres and DynamoDB.
- Functional and scripting languages: Python, Advanced SQL.