Azure Databricks Engineer - W2 only

The following requirement is open with our client.                           

Title                                     : Azure Databricks Engineer

Location                             : Louisville, KY (Onsite)

Pay rate                             : $45 - $49/hr on W2

Duration                            : 6 Months

Relevant Experience (in Yrs.): 8 yrs.

Strong hands-on experience with Azure Databricks including workspace setup cluster configuration job orchestration and integration with Azure services · Proficiency in Python PySpark and SQL for building scalable data pipelines and transformations · Familiarity with Unity Catalog Delta Lake and data lakehouse architecture including data governance and access control · Solid understanding of Azure Data Lake Storage and its integration with Databricks · Knowledge of GitHub for version control and CI CD pipeline implementation · Understanding of DevOps practices including automated deployment and environment management · Exposure to Azure Active Directory RBAC and networking principles for secure data platform access · Experience in data modeling including dimensional modeling and schema design for analytics and reporting · Familiarity with containerization and orchestration tools including Docker and Azure Kubernetes Service AKS

Roles & Responsibilities

· Develop scalable data pipelines using Databricks, Apache Spark, and related technologies.

· Collaborate with data scientists, analysts, and business stakeholders to understand requirements and deliver robust solutions.

· Optimize ETL workflows for performance, reliability, and cost efficiency on cloud platforms (Azure, AWS, or Google Cloud Platform).

· Implement data governance, security, and compliance best practices within the Databricks environment.

· Mentor junior engineers and contribute to code reviews, architectural decisions, and platform enhancements.

· Develop and maintain documentation for data pipelines, architecture, and operational procedures.

· Troubleshoot and resolve complex technical issues related to data ingestion, transformation, and storage.

Generic Managerial Skills, If any

· Exposure to Confluent Event Hub or similar event streaming platforms.

· Knowledge of DBT for data transformation and modeling in Lakehouse architecture.

· Prefect / terraform knowledge for workflow orchestration, monitoring, and infrastructure automation.

Thanks & regards, 

Naveen Yarramsetty

Technical Recruiter | ASCII Group, LLC
Office: 

Email: 

38345 W. 10 Mile Rd, Ste.#365; Farmington, MI  48335 

Website: 

Salary : $45 - $49