Data Engineer

Job Description:

This role will support the organization’s expanding cloud data platform and future enterprise analytics roadmap within Databricks. The Senior Data Engineer will work closely with internal business units and external engineering partners to support enterprise data pipelines, cloud database environments, operational analytics, system integrations, and future data development initiatives.

The position will play a key role in supporting current and future Databricks projects and sprint initiatives, including transportation and brokerage analytics, operational KPI reporting, driver payroll and activity-based pay analytics, telematics and reefer visibility integrations, customer visibility platforms, trailer and yard management integrations, AI and automation initiatives, and enterprise reporting solutions.

This role will support scalable cloud-based data solutions utilizing Databricks, SQL, Python, PySpark, AWS, Microsoft Azure, APIs, and modern ETL/ELT workflows across multiple operational systems and business units.

The ideal candidate will have experience working with large operational datasets, cloud data

environments, transportation management systems, telematics platforms, APIs, and modern analytics workflows within a fast-paced logistics and transportation environment.

Key Duties and Responsibilities:

• Develop and maintain scalable data solutions within Databricks.
• Build and optimize ETL/ELT workflows using SQL, Python, and PySpark.
• Support Bronze, Silver, and Gold layer data architecture and scalable analytics workflows.
• Optimize Spark workloads and data transformations for scalability and performance.
• Support ingestion workflows from APIs, operational systems, and external data providers.
• Work with structured and semi-structured datasets including JSON, CSV, and API responses.
• Support AWS-based cloud environments and Databricks integrations.
• Write and optimize advanced SQL queries and transformations.
• Develop Python scripts and Databricks notebooks for automation, validation, and operational analytics.

Required Qualifications:

• 5 years of experience in Data Engineering, Analytics Engineering, or related technical roles.
• Strong SQL skills required.
• Strong Python skills required.
• Experience with Databricks, PySpark, and Spark-based processing required.
• Experience working in AWS and Microsoft Azure cloud environments and modern ETL/ELT workflows.

Preferred Skills and Experience:

• Logistics / transportation industry experience preferred.
• Experience with Revenova, Samsara, and transportation operational datasets preferred.
• Bachelor’s degree in Computer Science, Data Analytics, Information Systems, Engineering, Mathematics, or related technical field preferred; equivalent experience accepted.
• Understanding of transportation workflows, APIs, JSON data structures, and operational analytics.
• Experience supporting hybrid AWS and Azure cloud architectures, integrations, and enterprise analytics environments preferred.
• Experience with Microsoft Azure services, Microsoft Entra ID (Azure AD), and enterprise identity/access management preferred.