Senior Data Engineer

Skills:

• Cloud Platforms: Deep understanding of Azure ecosystem, including Azure Data Factory, Data Lake Storage, Blob Storage, power apps, and Functions. Additionally, in-depth understanding and implementation of API management, such as Apigee.
• Big Data Technologies: Proficiency in Databricks, Spark, PySpark, Scala, and SQL.
• Data Engineering Fundamentals: Expertise in ETL/ELT processes, data pipelines, data modeling, schema design, and data warehousing.
• Programming Languages: Strong Python and SQL skills, with knowledge of other languages like Scala or R beneficial.
• Data Warehousing and Business Intelligence: Strong ERD concepts, designs, and patterns, understanding OLAP/OLTP systems, performance tuning, Database Server concepts, and BI tools (Power BI, Tableau).
• Data Governance: Strong understanding of RBAC/ABAC, Data Lineage, Data leak prevention, Data security, and compliance. Deep understanding and implementation knowledge of audit and monitoring in Cloud.
• Infrastructure Deployment: GitHub version control, CI/CD pipelines, release management, Terraform and YAML templates, and script-based deployments.

Experience Required:

Minimum five (5) years of hands-on experience in the following areas:

• Azure Data Factory, Azure Databricks, API implementation and management solutions, and Azure resource administration.
• Designing and developing data models and data pipelines using Python.
• Working with modern Lakehouse platforms and architectures.
• Managing GitHub CI/CD pipelines, infrastructure automation, and Terraform scripting.
• Data warehousing concepts and implementations.
• OLAP and OLTP systems.
• Integration of Business Intelligence (BI) tools and reporting platforms.
• Designing, developing, and deploying AI/ML and predictive analytics solutions using Databricks, Apache Spark, MLflow, Delta Lake, and Python.
• Experience with cloud platforms such as Microsoft Azure and/or AWS.
• Demonstrated expertise in building and operationalizing predictive models, Generative AI solutions, and enterprise-scale analytics pipelines to enable forecasting, operational intelligence, and data-driven decision-making.

Education Required:

• Bachelor’s degree in computer science or IT-related field.