KAFKA ADMINISTRATOR

We are seeking an experienced Kafka Administrator with strong expertise in Apache Kafka, Confluent technologies, AWS MSK, and large-scale production environments. The ideal candidate will have a background in Site Reliability Engineering (SRE) or DevOps, with hands-on experience managing highly available Kafka clusters, performing major cluster rebuilds and hardware refreshes, and executing migrations with minimal downtime.

Key Responsibilities

• Design, deploy, configure, and maintain highly available Apache Kafka clusters across on-premises, cloud, and managed environments, including AWS MSK and Confluent Cloud.
• Administer Kafka topics, partitions, quotas, retention policies, replication settings, and consumer groups to ensure optimal performance and cost efficiency.
• Manage Kafka ecosystem components, including Kafka Connect, Schema Registry, MirrorMaker, Confluent Replicator, and REST Proxy.
• Execute Kafka upgrades, patching, hardware refreshes, cluster rebuilds, and migrations while minimizing downtime and operational risks.
• Perform Kafka performance tuning, throughput testing, and benchmarking across producers, consumers, brokers, and controllers.
• Optimize batching, compression, acknowledgments (acks), in-sync replicas (ISR), and cluster health.
• Develop capacity plans, partitioning strategies, infrastructure sizing models, and cluster right-sizing recommendations.
• Integrate Apache Kafka with AWS MSK and other enterprise systems.
• Implement monitoring, alerting, troubleshooting, backup and recovery, and disaster recovery procedures.
• Apply strong Linux and networking fundamentals, including TCP/IP, DNS, and load balancing, to troubleshoot infrastructure and connectivity issues.
• Develop architecture diagrams, operational standards, runbooks, and migration documentation.
• Conduct knowledge transfer sessions and ensure smooth documentation handoff at key milestones and project completion.
  
  

Required Skills And Qualifications

• 8–10 years of overall relevant IT experience.
• 4 years of hands-on experience operating Kafka in production at scale within an SRE or DevOps environment.
• Strong experience with Apache Kafka administration and production operations.
• Hands-on experience with AWS MSK operations, configuration, and integration.
• Practical experience with Kafka Connect, Schema Registry, MirrorMaker, and Confluent Replicator.
• Strong Linux administration and networking fundamentals, including TCP/IP, DNS, and load balancing.
• Proven experience executing hardware refreshes, major cluster rebuilds, and migrations with minimal downtime.
• Experience with Kafka performance tuning, capacity planning, cluster sizing, and troubleshooting.
• Knowledge of high availability, replication, disaster recovery, monitoring, and operational best practices.
• Ability to create technical documentation, migration plans, architecture diagrams, and operational runbooks.
• Strong communication and collaboration skills.