What are the responsibilities and job description for the Senior Data Engineer position at Jobs via Dice?
Dice is the leading career destination for tech experts at every stage of their careers. Our client, Purple Drive Technologies LLC, is seeking the following. Apply via Dice today!
Job Title: Senior Data Engineer
Location: Irvine, CA
Job Type: Full-Time
Job Summary
We are seeking a highly skilled Senior Data Engineer with deep expertise in Databricks, Apache Spark, Airflow/Astronomer, and cloud-native data platforms. The ideal candidate will be responsible for designing, building, and supporting scalable data engineering solutions while ensuring platform reliability, performance, automation, and operational excellence.
This role requires hands-on engineering experience, strong DevOps capabilities, and the ability to independently own technical delivery in a highly collaborative engineering environment.
Required Skills
Job Title: Senior Data Engineer
Location: Irvine, CA
Job Type: Full-Time
Job Summary
We are seeking a highly skilled Senior Data Engineer with deep expertise in Databricks, Apache Spark, Airflow/Astronomer, and cloud-native data platforms. The ideal candidate will be responsible for designing, building, and supporting scalable data engineering solutions while ensuring platform reliability, performance, automation, and operational excellence.
This role requires hands-on engineering experience, strong DevOps capabilities, and the ability to independently own technical delivery in a highly collaborative engineering environment.
Required Skills
- Strong expertise in:
- Databricks
- Apache Spark
- Data Engineering
- Platform Engineering
- Hands-on experience with:
- Airflow
- Astronomer
- DAG Design & Orchestration
- Strong knowledge of:
- Data Pipeline Architecture
- Data Quality Frameworks
- Scalability & Performance Optimization
- Experience with:
- CI/CD Pipelines
- Infrastructure as Code (IaC)
- Automation Frameworks
- DevOps Best Practices
- Strong platform administration experience including:
- Monitoring
- Reliability Engineering
- Operational Readiness
- Incident Management
- Experience working with cloud-based data platforms
- Strong problem-solving and troubleshooting skills
- Design, develop, and maintain scalable data pipelines and data processing frameworks
- Build and optimize Databricks-based data engineering solutions
- Perform Spark performance tuning and platform optimization
- Develop and manage Airflow/Astronomer workflows and enterprise orchestration solutions
- Implement data quality, monitoring, and reliability frameworks
- Design and maintain CI/CD pipelines for data platform deployments
- Automate infrastructure provisioning and operational processes using Infrastructure as Code
- Ensure platform stability, availability, and operational excellence
- Collaborate with architects, data engineers, DevOps teams, and business stakeholders
- Troubleshoot and resolve complex platform and data engineering issues
- Drive engineering best practices, standards, and continuous improvement initiatives
- Experience supporting enterprise-scale data platforms
- Knowledge of cloud-native architecture and distributed systems
- Exposure to DataOps and Platform Engineering practices
- Strong communication and stakeholder management skills