Position: Azure Databricks Developer

Overall exp: 12 - 14 years

Location: Remote (Canonsburg, PA) Need to be there at client s location once in a month

Responsibilities

• 12 years of experience in implementing large data and analytics platforms.
• 5 years of experience architecting and building data platforms on Azure.
• 5 years of experience architecting and building data platforms on Databricks from scratch.
• Good Knowledge in Databricks engineering services like Unity Catalog, Spark Jobs, Delta Live Table (DLT), DLT Meta, Databricks Workflow, Auto Loader.
• Hands on in Databricks SQL, PySpark
• Understanding of data modelling, data mappings, understand the data model changes and propose the best practices and guidelines.
• Creating technical design based on end-to-end data architecture and frameworks for meta data driven data ingestion, transformation, logging and monitoring.
• Deploying data pipelines/applications to higher environments using CI-CD pipelines
• Experience in writing technical user stories.
• Experience in Agile methodology and scrums.

Minimum Qualifications

• Bachelor's Degree or Master's Degree in the field of Computer Science or Information Systems or a related field.

Preferred Qualifications

• Design Azure Databricks platform architecture from scratch.
• Design and build ingestion framework, data processing framework, logging & monitoring framework and, data sharing framework.
• Share Databricks best practices.
• Develop data models in different layers of Databricks.
• Design, build and orchestrate data pipelines in Azure and Databricks
• Define strategy for configuration management and CI/CD