What are the responsibilities and job description for the Azure Databricks / Python Data Engineer position at RAPSYS TECHNOLOGIES PTE LTD?
Must-Have Skills
- StrongAzure Databricks
- AdvancedPythondevelopment
- PySpark & Spark SQL
- Delta Lake & ADLS Gen2
- Strong ETL/ELT & Data Engineering experience
- Azure Data Factory (ADF)
- SQL & data pipeline development
- Data ingestion, quality, validation & governance
- Databricks job, cluster & workflow optimization
- Batch & near real-time data processing
- Hands-on experience withGenAI tools such as GitHub Copilot
- Strong software design, testing & CI/CD experience
- Git / GitHub Actions & Agile