What are the responsibilities and job description for the Data Engineer with Flink and Kafka position at LiveMindz?
Job Title: Data Engineer with Flink and Kafka
Location: Columbus, OH
Type: W2
Job Summary
We are looking for an experienced Data Engineer with strong expertise in Apache Flink and Apache Kafka to design, develop, and maintain scalable, real-time data processing pipelines. The candidate will work on high-volume streaming data applications, ensuring low-latency processing, data quality, fault tolerance, and reliable integration with downstream systems.
Key Responsibilities
- Design, develop, and maintain real-time data pipelines using Apache Flink and Apache Kafka.
- Build stream-processing applications using Flink DataStream API, Table API, and Flink SQL.
- Develop Kafka producers and consumers, manage topics and partitions, and optimize consumer groups.
- Implement event-time processing, watermarks, windowing, stateful computations, and complex event processing.
- Configure Flink checkpoints, savepoints, state backends, and recovery mechanisms.
- Implement reliable streaming pipelines with appropriate delivery guarantees and idempotent processing.
- Integrate Kafka and Flink with databases, data lakes, REST APIs, and downstream analytical platforms.
- Perform performance tuning, troubleshoot latency, monitor backpressure, and resolve production issues.
- Implement data validation, schema evolution, error handling, and dead-letter queue mechanisms.
- Collaborate with architects, DevOps teams, business analysts, and other engineering teams.
- Participate in code reviews, unit testing, integration testing, deployment, and production support.
- Follow security, governance, coding standards, and CI/CD best practices.
Salary : $60 - $65