Hadoop Admin

Job Title: Hadoop Administrator

Responsibilities

• Responsible for implementation and ongoing administration of Hadoop infrastructure.
• Responsible for cluster maintenance, troubleshooting, monitoring, and following proper backup & recovery strategies.
• Provisioning and managing the life cycle of multiple clusters like EMR & EKS.
• Infrastructure monitoring, logging & alerting with Prometheus, Grafana, and Splunk.
• Performance tuning of Hadoop clusters and Hadoop workloads, including capacity planning at the application/queue level.
• Responsible for memory management, queue allocation, and distribution experience in Hadoop/Cloud era environments.
• Should be able to scale clusters in production and have experience with 18/5 or 24/5 production environments.
• Monitor Hadoop cluster connectivity and security, including file system (HDFS) management and monitoring.
• Investigate and analyze new technical possibilities, tools, and techniques that reduce complexity, create more efficient and productive delivery processes, or deliver better technical solutions that increase business value.
• Involved in fixing issues, performing RCA, and suggesting solutions for infrastructure/service components.
• Responsible for meeting Service Level Agreement (SLA) targets and collaboratively ensuring team targets are met.
• Ensure all changes to production systems are planned and approved in accordance with the Change Management process.
• Collaborate with application teams to install operating system and Hadoop updates, patches, and version upgrades when required.
• Maintain central dashboards for all system, data, utilization, and availability metrics.

Ideal Candidate Profile

• Experience: 6–12 years of total experience, with at least 3 years of hands-on work in developing, maintaining, optimizing, and resolving issues in Hadoop clusters supporting business users.
• Operating Systems: Experience in Linux/Unix OS services, administration, shell, and awk scripting.
• Programming: Strong knowledge of at least one programming language — Python, Scala, Java, or R — with debugging skills.
• Hadoop Ecosystem: Experience in Hadoop components — MapReduce, Hive, Pig, Spark, Kafka, HBase, HDFS, H-Catalog, Zookeeper, and Oozie/Airflow.
• Security: Experience in Hadoop security including Kerberos, Knox, and TLS.
• Databases: Hands-on experience with SQL and NoSQL databases (HBase), including performance optimization.
• Tools & Automation: Experience in tool integration, automation, and configuration management using GIT and Jira platforms.
• Soft Skills: Excellent oral and written communication, presentation skills, and strong analytical and problem-solving abilities.