What are the responsibilities and job description for the Databricks Engineer - Need 10+ yrs. Exp position at Programmers.io?
Must Have:
- 5 years of experience in the IT/Technology and having atleast 3 years of experience in Azure Databricks
- Hands on experience in development of Azure cloud DWH & datalakes using Azure technologies like Spark (Scala or Python), ADF, Azure Databricks, Terraform & CI/CD.
- Strong programming skills in Pyspark, Python, writing complex queries in Postgresql, T-SQL
- 6 years of experience in coding complex U-SQL, Spark (Scala or Python). T-SQL, postgresql
- Should have experience in Azure Storages like ADLS Gen2, Azure Postgresql
- Design and develop automation to support continuous delivery and continuous integration process when it comes to cloud infrastructure.
- Work closely with the Development and Operations teams to build and maintain CI/CD pipelines to support automated deployment of infrastructure (DevOps) and Infrastructure as Code (IaC) using Terraform and Enterprise Github