Position summary

The Data Engineer plays a critical, hands-on role in Microsoft Fabric and data modernization at Central Technology Services. This role applies Microsoft Fabric, Python/PySpark and SQL skills to migrate legacy SQL Server datamarts into reliable, modern analytics solutions. The Data Engineer takes meaningful ownership of development and delivery, collaborating with experienced teammates and business experts to turn complex data into trusted insights.

Key Responsibilities

• Re-engineer legacy SQL Server datamarts in Fabric, working with business experts to understand rules and dependencies and deliver tested migrations, parallel runs and controlled cutovers.
• Develop reusable Python, PySpark and Spark SQL transformations in Fabric notebooks and Lakehouses; build reliable pipelines with dependencies, retries, monitoring and safe reruns.
• Tune Spark workloads for performance and efficient Fabric capacity use, including joins, shuffles, skew, partitioning and memory.
• Implement incremental ingestion, change capture, upserts, schema evolution and backfills; build BI-ready dimensional models with clear grain, keys and history.
• Protect data accuracy with automated testing, quality checks and source-to-target reconciliation; troubleshoot production issues and improve reliability.
• Use Git, code review and CI/CD for deployment across environments; apply access controls, secure connectivity and data-protection practices with platform and security colleagues.
• Provide pragmatic support for selected legacy workloads during transition; Microsoft SQL Data marts.

Qualifications:

Required qualifications

• 5 years of relevant data engineering or development experience, or equivalent demonstrated capability, with strong production Python/PySpark and SQL development skills.
• Bachelor’s degree in Computer Science, Engineering or a related field, or equivalent relevant work experience.
• Hands-on Microsoft Fabric experience with Lakehouses, OneLake/Delta tables, notebooks and pipelines, plus production Python, PySpark and Spark SQL transformation, troubleshooting and performance tuning.
• Strong SQL Server and T-SQL skills, including stored procedures, execution-plan analysis and indexing; ability to translate existing business logic into modern pipelines.
• Practical dimensional-modeling skills and experience with incremental processing, schema changes, automated testing, data-quality controls and migration reconciliation.
• Proficiency with Git, CI/CD and environment-based deployment; practical knowledge of the on-premises data gateway for secure source connectivity.
• Ability to independently develop, test, deploy and troubleshoot assigned work within team architecture, seek design review and communicate clearly with technical and business colleagues.
• Sound judgment in protecting sensitive data and applying privacy, security and regulatory safeguards; commitment to documentation and knowledge sharing.

Preferred Qualifications

• Experience migrating SQL Server datamarts or transferable Spark experience from Microsoft Fabric, Databricks, or Azure Synapse.
• Advanced Power BI semantic-model development, including DAX, row-level security and Direct Lake.
• Banking or financial-services experience, or relevant Microsoft Fabric or data engineering certifications. Certifications are not required.