What are the responsibilities and job description for the Kafka Administrator position at Smart Folks Inc.?
Role: Kafka Administrator
Location - Jersey City, NJ/ Dallas, TX – onsite (F2F interview)
Duration: 6 months
Role Descriptions:
A Kafka Administrator is responsible for installing| configuring| maintaining| and monitoring Apache Kafka clusters to ensure high availability| security| and optimal performance.
They manage topics| partitions| and replication| troubleshoot issues| implement security protocols (SSLTLS| SASL| ACLs)| and automate tasks using tools like Ansible or Terraform.
Roles and Responsibilities
Cluster Management Installing| configuring| and upgrading Kafka brokers| ZookeeperKRaft| Connect| and Schema Registry.
Performance Tuning Monitoring cluster health| broker performance| consumer lag| and producer latency using tools like Prometheus and Grafana to ensure high throughput.
Topic Partition Management Managing topic configuration| partitioning strategies| replication factors| and data retention policies.
Security Access Control Implementing security measures such as SSLTLS encryption| Kerberos authentication| and ACLs (Access Control Lists) to secure data.
Troubleshooting Support Providing 3rd-level support to resolve broker| producer| or consumer issues and performing root cause analysis (RCA).
Disaster Recovery High Availability Designing and managing replication strategies across regions for disaster recovery and ensuring broker redundancy.
Automation Developing scripts to automate routine tasks| such as provisioning and monitoring| using Ansible| Puppet| or Kubernetes.
Collaboration Working with developers to design data pipelines| optimize producer consumer configurations| and onboard new applications.
Required Skills:
In-depth knowledge of Apache Kafka, Confluent Kafka.
Strong Linux system administration skills.
Experience with monitoring tools (Prometheus| Grafana| JMX).
Scripting proficiency in Python| Bash| or Java.
Experience with containerization tools (Docker| Kubernetes OpenShift).