What are the responsibilities and job description for the Azure Databricks Data Engineer position at First Soft Solutions LLC?
Work Authorization: U.S. Citizens Only (No Sponsorship)
Job Summary
We are seeking an experienced Senior Azure Databricks Data Engineer to join a large-scale enterprise data modernization initiative. The ideal candidate will have extensive hands-on experience designing, developing, and optimizing cloud-based data engineering solutions using Azure Databricks and the Microsoft Azure ecosystem. This role requires strong expertise in building scalable data pipelines, implementing modern Lakehouse architectures, integrating enterprise data sources, and delivering high-performance analytics solutions while collaborating with cross-functional teams in an Agile environment.
Key Responsibilities
Design, develop, and maintain enterprise-scale data pipelines using Azure Databricks, Azure Data Factory (ADF), PySpark, Spark SQL, Python, Azure Data Lake Storage Gen2 (ADLS Gen2), Delta Lake, Lakehouse Architecture, Medallion Architecture (Bronze, Silver, Gold), Azure Synapse Analytics, Microsoft Fabric, Azure SQL Database, SQL Server, SQL, ETL/ELT, Change Data Capture (CDC), Incremental Data Loading, REST API Integration, Data Modeling, Data Warehousing, Star Schema, Snowflake Schema, Power BI, Tableau, Data Governance, Data Quality, Metadata Management, Performance Tuning, Query Optimization, Git, Azure DevOps, CI/CD Pipelines, Agile/Scrum, and enterprise cloud technologies. Collaborate with architects, developers, business analysts, and stakeholders to build scalable, secure, high-performing data platforms while ensuring data integrity, governance, monitoring, troubleshooting, documentation, and continuous improvement of enterprise analytics solutions.
Required Qualifications
Job Summary
We are seeking an experienced Senior Azure Databricks Data Engineer to join a large-scale enterprise data modernization initiative. The ideal candidate will have extensive hands-on experience designing, developing, and optimizing cloud-based data engineering solutions using Azure Databricks and the Microsoft Azure ecosystem. This role requires strong expertise in building scalable data pipelines, implementing modern Lakehouse architectures, integrating enterprise data sources, and delivering high-performance analytics solutions while collaborating with cross-functional teams in an Agile environment.
Key Responsibilities
Design, develop, and maintain enterprise-scale data pipelines using Azure Databricks, Azure Data Factory (ADF), PySpark, Spark SQL, Python, Azure Data Lake Storage Gen2 (ADLS Gen2), Delta Lake, Lakehouse Architecture, Medallion Architecture (Bronze, Silver, Gold), Azure Synapse Analytics, Microsoft Fabric, Azure SQL Database, SQL Server, SQL, ETL/ELT, Change Data Capture (CDC), Incremental Data Loading, REST API Integration, Data Modeling, Data Warehousing, Star Schema, Snowflake Schema, Power BI, Tableau, Data Governance, Data Quality, Metadata Management, Performance Tuning, Query Optimization, Git, Azure DevOps, CI/CD Pipelines, Agile/Scrum, and enterprise cloud technologies. Collaborate with architects, developers, business analysts, and stakeholders to build scalable, secure, high-performing data platforms while ensuring data integrity, governance, monitoring, troubleshooting, documentation, and continuous improvement of enterprise analytics solutions.
Required Qualifications
- Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
- 5 years of Data Engineering experience with 3 years of hands-on experience in Azure Databricks.
- Strong hands-on experience with Azure Databricks, PySpark, Spark SQL, Python, Azure Data Factory (ADF), Azure Data Lake Storage Gen2 (ADLS Gen2), Delta Lake, Lakehouse Architecture, Medallion Architecture, Azure Synapse Analytics, Microsoft Fabric, Azure SQL Database, SQL Server, Advanced SQL, ETL/ELT Development, Data Warehousing, Data Modeling, Star/Snowflake Schema, Incremental Data Loading, Change Data Capture (CDC), REST API Integration, Power BI, Tableau, Git, Azure DevOps, CI/CD Pipelines, Performance Tuning, Query Optimization, Data Governance, Data Quality, Metadata Management, and experience working within Agile/Scrum environments.
- Excellent analytical, troubleshooting, communication, and problem-solving skills.
- Ability to work independently and collaboratively in a fast-paced enterprise environment.
- U.S. Citizenship is required. No Sponsorship.