What are the responsibilities and job description for the Azure Databricks Engineer position at Sonitalent Corp?
Job Description
Responsibilities
- Design, develop, and maintain scalable ETL pipelines using Azure Databricks
- Build resilient data ingestion processes from on-premise and cloud-based transactional databases and APIs
- Develop reusable and efficient workflows using Databricks Notebooks and Delta Live Tables
- Collaborate with technical leads and development teams for solution design and implementation
- Work with business stakeholders to gather requirements and support change management activities
- Develop batch and streaming data processing solutions using Apache Spark
- Collaborate with DBAs and business users to create high-quality data products
- Ensure data quality, validation, governance, and security best practices
- Maintain technical documentation for data engineering processes
- Support enterprise data architecture initiatives and modernization efforts
Required Skills
- 5 years of excellent verbal and written communication skills
- 2 years of experience writing clean and maintainable Databricks Notebook code
- Strong understanding of:
- Data Engineering Best Practices
- Data Warehouses & Data Lakes
- Delta Lake Architecture
- Hands-on experience with:
- Azure Databricks
- Spark / PySpark
- Databricks SQL
- Delta Live Tables
- Experience working with cloud platforms such as:
- Azure
- AWS
- Google Cloud Platform (GCP)
- Strong knowledge of:
- OLTP & OLAP systems
- Apache Spark
- Streaming technologies like Azure Service Bus
- Experience building scalable ETL and streaming solutions
Desired Skills
- Experience with object-oriented programming languages such as:
- Python
- Java
- C#
- Prior experience working with State or Government clients preferred
Required Skills: Python, Java, C#,Azure, AWS, Google Cloud Platform,OLTP & OLAP systems Apache Spark