What are the responsibilities and job description for the Data Engineer position at PROLIM Global Corporation?
We are seeking a Data Engineer with strong experience in Azure Databricks, Power BI, Python, and enterprise data pipelines. The ideal candidate will be responsible for designing, developing, and maintaining scalable data solutions that support analytics, reporting, and business intelligence initiatives.
Required Skills
- Azure Databricks (PySpark, Spark SQL)
- Power BI
- Python
- Azure Data Factory (ADF)
- ETL / ELT Development
- Data Modeling
- SQL & NoSQL Databases
- Azure Data Lake / Delta Tables
- Medallion Architecture
- CI/CD for Data Engineering
Key Responsibilities
- Design, develop, and maintain end-to-end data pipelines using Azure Databricks and Azure Data Factory.
- Build scalable batch and incremental data ingestion frameworks from APIs, databases, files, and cloud storage.
- Develop and optimize data transformation logic using PySpark, Spark SQL, and Python.
- Create and maintain data models for analytics, reporting, and machine learning initiatives.
- Enable Power BI datasets and reporting solutions for business stakeholders.
- Implement data quality checks, monitoring, and validation frameworks.
- Optimize pipeline performance, storage, and compute resources in Azure.
- Support data governance, lineage, cataloging, and documentation initiatives.
- Collaborate with cross-functional teams to translate business requirements into technical solutions.
- Support CI/CD deployment processes for data engineering workflows.
Minimum Qualifications
- 5 years of experience in Data Engineering.
- Strong hands-on experience with Azure Data Factory and ETL/ELT development.
- Proven experience with Azure Databricks (PySpark/Spark SQL).
- Experience with Medallion Architecture and Delta Tables.
- Strong SQL skills and experience with relational and NoSQL databases.
- Knowledge of Data Lake and Data Warehouse architectures.
- Experience working with large-scale structured and semi-structured datasets.
- Excellent communication and collaboration skills.
Salary : $50 - $55