What are the responsibilities and job description for the Java Developer position at Avance Consulting?
Perform for the end-to-end datastore migration from on-prem Data-Lake to AWS hosted Lake-House
Engineer will be part of the datastore-migration Factory team that will be responsible to perform for the end-to-end datastore migration from on-prem DataLake to AWS hosted LakeHouse. This is a high visibility and crucial project for the client.
Responsibilities of the Engineer includes:
Skill set one:-
Hands on programming Python, Pyspark.
Skills:- Apache Spark, Hadoop (HDFS/Hive), Sybase IQ, Shell scripting , Apache Iceberg , AWS Ecosystems for bigdata processing ( EMR, Redshift, Athena, Glue eic ), AWS Lamda expressions ,Step functions, CI/CD best practices & Kubernates deployment experience
Skill set two:-
Hands on programming JAVA, Python
Skills:- Snowflake, Kafka, SnowSQL, Apache Iceberg, Shell scripting , Apache Iceberg , AWS Ecosystems for bigdata processing ( EMR, Redshift, Athena, Glue eic ), AWS Lamda expressions ,Step functions, Data moulding experience (SCD Type 2), CI/CD best practices & Kubernates deployment experience