What are the responsibilities and job description for the Lead Big Data Engineer position at Mogi I/O : OTT/Podcast/Short Video Apps for you?
Skills: Azure Databricks, Apache Spark/PySpark, Azure Data Factory, Azure Functions, Delta Lake, ADLS Gen2, SQL, ETL/ELT pipelines, Data Lakehouse architecture, Data Modeling,
Location: Alpharetta, Georgia
Experience Required: 3 – 15 Years
Compensation: USD 82000 - 123000
Visa Status: US Citizens, green card holder
About The Role
We are seeking a Senior/Lead Data Engineer with 8 years of experience building scalable, high-performance data platforms on Azure & Databricks. This role involves end-to-end data engineering, architecture, and leadership across modern data lakehouse environments.
Key Responsibilities
Location: Alpharetta, Georgia
Experience Required: 3 – 15 Years
Compensation: USD 82000 - 123000
Visa Status: US Citizens, green card holder
About The Role
We are seeking a Senior/Lead Data Engineer with 8 years of experience building scalable, high-performance data platforms on Azure & Databricks. This role involves end-to-end data engineering, architecture, and leadership across modern data lakehouse environments.
Key Responsibilities
- Architect and implement scalable data platforms and pipelines on Azure and Databricks.
- Build and optimize batch & real-time data ingestion and transformation (Spark/PySpark).
- Develop robust ETL/ELT workflows, ensuring reliability and performance.
- Work with ADLS, Delta Lake, and Spark for large-scale processing.
- Design data models (conceptual, logical, physical) for analytics and operational use.
- Collaborate with cross-functional teams to convert requirements into technical solutions.
- Troubleshoot performance issues and support production systems.
- Ensure best practices for data governance, security, quality, and architecture.
- Mentor junior engineers and contribute to engineering standards.
- 8 years of enterprise data engineering experience.
- Strong hands-on experience with:
- Azure Databricks, Azure Functions, ADF
- Apache Spark (PySpark)
- SQL & performance tuning
- Delta Lake (schema evolution, ACID, optimization)
- Expertise in ETL/ELT, distributed systems, and cloud-native architecture.
- Experience with data modelling (dimensional, normalized, lakehouse).
- Exposure to streaming frameworks (Kafka, Event Hub).
- Strong problem-solving and technical leadership skills.
- CI/CD using Azure DevOps, Git.
- IaC tools (Terraform/ARM).
- Azure Purview or other data governance tools.
- Experience supporting ML or BI workloads on Databricks.
- Medical, dental, vision insurance
- Paid vacation, sick leave, holidays
- Employer-matched 401(k)
- Life, AD&D, short-term & long-term disability
- Eligibility varies by employment type and state law
Salary : $82,000 - $123,000