What are the responsibilities and job description for the Lead Databricks/Spark Engineer position at InfiCare?
Job Title: Lead Databricks/Spark Engineer
Location: Columbus, GA or Atlanta, GA (Onsite)
Job Type: Contract
Overview
The ideal candidate will have deep hands-on expertise with Apache Spark, Databricks, and Python, and experience working on large-scale data platforms. This role involves understanding existing applications and frameworks, resolving migration challenges, optimizing Spark workloads, and modernizing solutions on Databricks.
Required Skills
- Strong Apache Spark / PySpark experience
- Hands-on Databricks experience
- Python programming
- Spark performance tuning and troubleshooting
- Batch and streaming data processing
- Data platform modernization or migration experience
- Strong debugging and problem-solving skills