What are the responsibilities and job description for the Azure data engineer position at Compunnel Inc.?
Title: Azure Data Engineer
Location: Onsite role in Charlotte, NC
Duration : Long Term
Job description
Responsibilities:
- 10 Years experience with ADF, Databricks and Azure Functions (C#)
- In depth development experience python/notebook on Databricks/Spark/panda
- Expert working knowledge of Unity Catalog and data governance setup/configuration
- Experience in loading datasets from various sources using ADF to Databricks with Medallion architecture
- Experience on converting Databricks workspace from hive metastore to unity catalog & setup associated permissions
- Creating delta table schema/tables with proper design/properties & security permissions
- Nice to have experience on Delta Live Table.
- DevOps & deployment pipeline using Azure DevOps/GitHub Enterprise (especially on databricks on cluster/secret management and code)
- In depth experience on Azure Entra ID for SSO/RBAC setup
- In depth experience on working with/deploying to Azure PaaS resources such as Azure SQL PaaS, ADF/SHIR, ADLS, Databricks, Synapse dedicated SQL databases
- Familiar to work in lock down windows environment for highly regulated industry
- Ability to work independently and lead a small team.
- Proven ability to work independently with minimum supervision
- Ability to lead a small team and technical problem-solving sessions
- Financial Services Industry experience is an Asset.
- Excellent Communication skills