What are the responsibilities and job description for the Lead Data Engineer position at Flexton Inc.?
The team is seeking a Data Engineer experienced in implementing modern data solutions in Azure, with strong hands-on skills in Databricks, Spark, Python, and cloud-based DataOps practices. The Data Engineer will analyze, design, and develop data products, pipelines, and information architecture deliverables, focusing on data as an enterprise asset. This role also supports cloud infrastructure automation and CI/CD using Terraform, GitHub, and GitHub Actions to deliver scalable, reliable, and secure data solutions.
Requirements
• 5 years of experience as a Data Engineer
• Hands-on experience with Azure Databricks, Spark, and Python
• Experience with Delta Live Tables (DLT) or Databricks SQL
• Strong SQL and database background
• Experience with Azure Functions, messaging services, or orchestration tools
• Familiarity with data governance, lineage, or cataloging tools (e.g., Purview, Unity Catalog)
• Experience monitoring and optimizing Databricks clusters or workflows
• Experience working with Azure cloud data services and understanding how they integrate with Databricks and enterprise data platforms
• Experience with Terraform for cloud infrastructure provisioning
• Experience with GitHub and GitHub Actions for version control and CI/CD automation
• Strong understanding of distributed computing concepts (partitions, joins, shuffles, cluster behavior)
• Familiarity with SDLC and modern engineering practices
• Ability to balance multiple priorities, work independently, and stay organized