What are the responsibilities and job description for the Big Data Engineer position at ECS Resource Group?
Big Data Engineer – Hadoop
Hybrid Working in Austin Office
W2 Contract
ECS Resource Group is seeking an experienced Big Data Engineer - Hadoop to support a large-scale, open-source Big Data platform; the role focuses on ensuring the availability, stability, and operational performance of Hadoop environments.
Key Responsibilities:
- Operate and support Hadoop platforms (HDFS, YARN, MapReduce, Spark, HBase) across production and non-production environments.
- Perform cluster maintenance, upgrades, patching, and configuration changes while ensuring platform stability.
- Troubleshoot complex platform and application issues, conduct root cause analysis, and implement preventative measures.
- Automate operational tasks using Ansible, Python, and shell scripting to improve efficiency and reliability.
- Collaborate with engineering and data teams to enhance platform resilience, monitoring, and operational excellence.
Required Skills:
- Strong hands-on experience with Hadoop ecosystems and distributed systems.
- Deep knowledge of HDFS, YARN, MapReduce, Spark, and HBase.
- Solid Linux/Unix system skills and understanding of networking concepts.
- Experience supporting critical production within a large enterprise environment.
Please contract Olivia@ecsgrp.co.uk / 01676545407