Data Engineer - Azure & Microsoft Fabric Platform

Job Summary:  

PGW Auto Glass (PGWAG) is seeking a highly motivated Data Engineer to help modernize and scale our enterprise analytics platform on Microsoft Azure and Microsoft Fabric. This role will focus on designing, developing, and maintaining cloud-native data engineering pipelines that support enterprise reporting, real-time analytics, AI/ML initiatives, pricing optimization, and operational intelligence across our network of branches and distribution centers throughout the United States and Canada. 

The ideal candidate will possess strong experience in Azure-based data engineering, real-time event streaming, batch processing, Lakehouse architectures, and modern analytics platforms. This role sits at the intersection of Data Engineering, Cloud Architecture, Real-Time Analytics, and AI enablement. 

The candidate will work closely with Pricing, Supply Chain, Operations, IT Infrastructure, and Executive Leadership teams to build scalable and resilient analytics solutions using Microsoft Fabric, Azure Databricks, Event Streaming technologies, and Power BI. 

Role Overview 

We are seeking a mid-level Data Engineer to design, scale, and maintain our dual-engine enterprise data platform on Microsoft Azure and Microsoft Fabric. This role balances both batch and real-time processing architectures, ensuring seamless data flow from transactional systems into analytics-ready storage and semantic models. 

This position is critical to PGWAG’s Cloud Modernization, Data Warehouse Modernization, and AI enablement initiatives. 

Key Responsibilities & Duties:  

Microsoft Fabric & Azure Data Engineering 

• Design, develop, and maintain scalable enterprise data pipelines using: 

o Microsoft Fabric 

o Azure Data Factory 

o Fabric Data Factory 

o Azure Databricks 

o Azure Event Hubs 

o OneLake 

o Fabric Lakehouse 

o Fabric Data Warehouse 

• Build analytics-ready datasets supporting: 

o Pricing Analytics 

o Supply Chain Analytics 

o POS Sales Analytics 

o Customer Behavior Analytics 

o Executive Dashboards 

o AI/ML workloads 

Dual-Engine Data Pipelines 

• Build and manage parallel processing architectures using: 

o Azure Data Factory for structured batch processing 

o Azure Event Hubs / Kafka for real-time event ingestion 

• Support ingestion patterns including: 

o Batch ETL/ELT 

o Change Data Capture (CDC) / Database mirroring 

o Streaming ingestion 

o API-based integrations 

o SaaS integrations 

•  Develop near real-time analytics solutions using Eventstream and Real-Time Intelligence capabilities in Microsoft Fabric. 

Stream & Batch Processing 

• Develop and optimize PySpark workloads using: 

o Azure Databricks 

o Fabric Spark 

o Spark Structured Streaming 

• Process: 

o High-volume historical datasets 

o XML/JSON log files 

o Streaming transactional events 

o Operational telemetry data 

• Build scalable transformation logic for both streaming and batch architecture. 

Data Modeling & Transformation 

• Model and transform enterprise data using: 

o ANSI SQL 

o T-SQL 

o dbt (Data Build Tool) 

o Lakehouse design principles 

• Design: 

o Star schemas 

o Snowflake schemas 

o Semantic models 

o Curated analytical datasets 

• Support enterprise-wide self-service analytics initiatives using governed semantic layers. 

Storage & Lakehouse Architecture 

•  Maintain scalable Azure Data Lake Storage (ADLS Gen2) environments. 
• Implement and optimize: 

o Delta Lake table formats 

o ACID-compliant storage patterns 

o Schema evolution and enforcement 

o Partitioning and performance tuning 

• Support enterprise Lakehouse architecture using Microsoft Fabric OneLake. 

Power BI & Analytics Enablement 

• Partner with Analytics and Business teams to deliver: 

o Power BI dashboards 

o Executive scorecards 

o KPI reporting 

o Self-service analytics solutions 

• Build and maintain: 

o Semantic models 

o Direct Lake datasets 

o Row-level security 

o Data governance standards 

• Support Copilot-enabled analytics and AI-assisted reporting capabilities. 

Infrastructure, Automation & DevOps 

• Deploy and maintain cloud infrastructure using: 

o Terraform 

o Azure Resource Manager (ARM) 

o Infrastructure-as-Code principles 

• Automate CI/CD workflows using: 

o Azure DevOps 

o Git 

o Docker 

• Author and orchestrate enterprise workflows using: 

o Azure Data Factory 

o Fabric Pipelines 

o Managed Apache Airflow 

o Control-M integrations where applicable 

Data Observability & Reliability 

• Implement automated monitoring and alerting for: 

o Batch failures 

o Streaming interruptions 

o Data quality issues 

o Schema drift 

o Pipeline latency 

• Build checksum and reconciliation frameworks between source systems and analytics platforms. 
• Support enterprise data governance and operational resiliency initiatives. 

Qualifications & Skills:      

Required Technical Skills 

Cloud & Data Platforms 

• Microsoft Azure 
• Microsoft Fabric 
• Azure Data Lake Storage Gen2 (ADLS Gen2) 
• Azure Databricks 
• Azure Data Factory 
• Azure Event Hubs 
• Azure Synapse Analytics / Fabric Warehouse 

Programming & Query Languages 

• Python 
•  PySpark 
• ANSI SQL 
• T-SQL 

Streaming & Batch Technologies 

• Apache Spark Structured Streaming 
• Apache Kafka 
• Azure Stream Analytics 
• Event-driven architectures 

Data Transformation & Storage 

•  dbt (Data Build Tool) 
• Delta Lake 
• Lakehouse architecture 
• Data warehousing concepts 

Data Modeling 

• Star Schema 
• Snowflake Schema 
• Semantic Layer Design 
• Enterprise Data Modeling 

Preferred Qualifications 

•  Bachelor’s or Master’s degree in: 

o Data Science 

o Computer Science 

o Information Systems 

o Engineering 

o Statistics 

o Mathematics 

o or related field 

• 3–5 years of experience in: 

o Data Engineering 

o Cloud Analytics 

o Enterprise Data Platforms 

o Real-Time Data Processing 

• Experience with: 

o Power BI Analytics 

o Fabric Real-Time Intelligence 

o OneLake 

o REST APIs 

o XML/JSON processing 

o Event-driven architecture 

• Exposure to: 

o AI/ML workloads 

o Azure OpenAI 

o Copilot integrations 

o Predictive analytics 

• Experience supporting enterprise analytics environments with large-scale operational datasets. 

Soft Skills 

• Strong analytical and problem-solving mindset 
• Excellent communication and collaboration skills 
• Ability to work across technical and business teams 
• Strong organizational and prioritization skills 
• Ability to manage multiple initiatives simultaneously 
• Curiosity and passion for emerging cloud and AI technologies 

Work Environment & Physical Requirements (if applicable): 

PGW Auto Glass, LLC is working in a hybrid environment.  Team members are expected to commute to our Cranberry Township, PA, Headquarters three days per week. Visa sponsorship is not available for this role.    

Why Join PGW Auto Glass? 

• Opportunity to help modernize enterprise analytics using Microsoft Fabric and Azure AI technologies 
• Exposure to cutting-edge cloud-native data engineering architectures 
• Work on highly visible enterprise initiatives with executive sponsorship 
• Opportunity to shape the future of AI-enabled analytics at PGWAG 
• Collaborative environment with strong growth and innovation opportunities 

Benefits and Compensation: 

• Comprehensive health, dental, vision, and disability coverage options. 
• Employer-provided life insurance and long-term disability benefits. 
• Paid time off (PTO) and paid holidays. 
• 401(k) retirement plan with company match. 
• Parental leave and support continuing education.