What are the responsibilities and job description for the Azure Databricks Data Engineer position at Prama Innovations India Pvt. Ltd.?
Job Overview
We are seeking an experienced Azure Databricks Data Engineer to design, develop, and optimize enterprise-scale data solutions on the Databricks Lakehouse Platform. The ideal candidate will have strong expertise in Azure Data Engineering, Databricks, Spark/PySpark, Delta Lake, and modern cloud-based data architectures.
Azure Databricks Data Engineer
Location: Warren, MI (Hybrid)
Duration: Long-Term Contract
Rate: Up to $50/hr C2C
Required Experience
- 5 years of Data Engineering experience
- 5 years of Azure Data Engineering experience
- 4 years of hands-on Azure Databricks experience
- Strong experience with Apache Spark / PySpark
- Expertise in Python and SQL
- Experience building ETL/ELT pipelines
- Strong knowledge of Delta Lake and Lakehouse architecture
- Experience with Data Modeling and Data Warehousing
- Cloud experience in Azure (preferred)
Key Responsibilities
- Design and maintain scalable data pipelines using Azure Databricks and Spark.
- Develop and optimize ETL/ELT workflows for large-scale datasets.
- Build and manage Lakehouse-based data solutions.
- Implement data quality, governance, and security best practices.
- Optimize Spark workloads for performance and cost efficiency.
- Integrate structured and unstructured data from multiple sources.
- Support CI/CD processes and infrastructure automation.
- Collaborate with Analytics, BI, AI/ML, and business teams.
Preferred Skills
- Unity Catalog
- Azure DevOps / GitHub Actions
- Terraform
- Kafka / Azure Event Hubs
- Snowflake
- Power BI
- Microsoft Fabric
- Azure Purview
- Generative AI and RAG architecture exposure
Certifications (Preferred)
Microsoft Azure Data Engineer Associate (DP-203)
Databricks Certified Data Engineer
Salary : $45 - $50