Data Engineer (Spark, Databricks, Azure, Scala & Java)

Data Engineer (Spark, Databricks, Azure, Scala & Java)

Job Type- Full Time

We are looking for a hands-on Data Engineer (Spark, Databricks, Azure, Scala & Java) for one of our direct client which is in to Asset Management and Investment Banking.

RESPONSIBILITIES;

• implement and deliver high quality software solutions for our Non-Financial Risk Metrics Assessment Tool and Workbench platforms. 

• help us build state of the art solutions that underpin GCRG using cutting edge technologies like Scala, Databricks, Java, Azure Data Lake, Apache Spark, Spring Boot, ReactJS, Kubernetes, Python, Postgres etc.

• demonstrate superior collaboration skills in working closely with other development, testing and implementation teams to roll-out important regulatory and business improvement programs

• develop required web services by analyzing business functional/technical requirements and implement them

• support the technical development and ensure delivery of well tested, signed-off and fit-for-purpose solutions

• ability to review stories / requirements, design solutions, and develop software components within agile delivery framework

• collaborate with other global IT and cross-functional teams as group moves towards agile delivery solutions

Your expertise

• a degree level education; preferably Computer Science (Bachelor or Master’s degree)

• ideally 8 years of hands-on design and development experience in several of the relevant technology areas, preferably in a cloud environment (Lakehouse architecture, Azure Data Lake, Scala, Databricks, Spark/Spark SQL, Spring Boot, ReactJS, Kubernetes, Postgres) 

• ideally 3 years of hand-on experience in distributed processing using Databricks, Apache Python/Spark, Kafka & leveraging Airflow scheduler/executor framework

• ideally 2 years of hand-on experience programming experience in Scala (must have), Python & Java (preferred)

• experience working with Agile development methodologies and delivering within Azure DevOps, automated testing on tools used to support CI and release management

• coding skills in Java, SQL, Spring Boot and other Java based frameworks (added advantage)

• coding skills in UI development using ReactJS (Optional)

• demonstrate strong communication skills, both to management and teams

• background in compliance and Risk Management

• collaborative approach towards problem solving, working closely with other colleagues in the global team and sensitive towards diversity