What are the responsibilities and job description for the Lead Big Data Engineer position at Programmers.io?
We are seeking an experienced Big Data Lead to drive the design, development, optimization, and maintenance of large-scale data platforms and Big Data solutions. The ideal candidate will have hands-on expertise with distributed computing technologies, data engineering, cloud platforms, and leading technical teams. This role requires strong architecture and development skills, excellent communication, and the ability to work onsite with cross-functional business and technology teams in Tampa, FL.
Job Title: Big Data Lead
Location: Tampa, FL (Onsite)
Employment Type: Full-time with all the benefits
Required Skills & Experience
- 10 years in Data Engineering / Big Data technologies.
- 4 years leading data engineering or Big Data teams.
- Expertise in:
- Hadoop ecosystem (HDFS, Hive, HBase, MapReduce, YARN)
- Spark (PySpark/Scala)
- Kafka or other streaming platforms
- Airflow / NiFi / Oozie
- SQL, NoSQL databases
- Strong cloud experience in AWS / Azure / GCP (S3, EMR, Databricks, BigQuery, ADLS, etc.).
- Proficient in Python, Scala, or Java.
- Experience with containerization (Docker, Kubernetes) is a plus.
- Solid understanding of data modeling, data warehousing, and distributed systems.