What are the responsibilities and job description for the Senior Data Engineer position at Hashlist?
We're looking for a Senior Data Engineer to design, build, and optimise robust data pipelines that process massive datasets in both batch and real-time. Working at the intersection of software engineering and data science, you'll ensure our data architecture is scalable, reliable, and follows industry best practices - building systems that power B2B data products at global scale.
Engagement detail:
- Location: Auburn Hills
- Contract type: Permanent/Contract
- Start date: ASAP
- Work model: Hybrid
- Benefits: Competitive package; high-impact, high-visibility role within a major software-defined vehicle programme
Responsibilities:
- Design and implement complex data processing pipelines using Apache Spark
- Build scalable, distributed systems for high-throughput data streams and large-scale batch processing
- Design and implement real-time pipelines with strong data quality and validation controls
- Manage and provision cloud infrastructure using Terraform
- Implement CI/CD workflows using GitHub Actions for automated testing and deployment
- Uphold rigorous engineering standards — unit/integration testing, code reviews, and clean documentation
- Translate business requirements into technical specifications alongside cross-functional stakeholders
Qualifications:
- 5 years in data engineering and the software development lifecycle
- 4 years hands-on with AWS cloud services
- 4 years building production data applications across relational and columnar data stores
- Deep expertise in Apache Spark — Core, SQL, and Structured Streaming
- Strong proficiency in Scala or Java for JVM-based production development
- Advanced SQL for transformation, analysis, and performance tuning
- Hands-on experience with Terraform and GitHub Actions
- Experience with Apache Flink for low-latency stream processing
- Proficiency in Python for automation and scripting
- Familiarity with Kafka, event-driven architectures, and streaming data
- Experience with workflow engines — Airflow, Luigi, or Azure Data Factory
- Familiarity with Lakehouse architectures (Delta Lake, Iceberg) or NoSQL databases
- Degree in Computer Science, Engineering, Mathematics, or related field preferred
Next Steps
- Press "Apply"We will review your application
- If qualified, you will be accepted into the network and can be considered for this and similar positions