What are the responsibilities and job description for the Data Integration Engineering Lead position at Pinnacle?
Pinnacle Reliability is an enterprise SaaS company serving the world’s leading oil and gas, refining, and chemical companies. Our platform helps clients manage asset reliability, optimize maintenance strategies, and make data-driven decisions about their critical equipment. Our customers include major operators across North America, Europe, and Asia-Pacific.
What You’ll DoSource System Extraction (This Is the Core of the Role)• Independently extract data from industrial source systems including OSIsoft PI historians, SAP PM/EAM, Maximo, eMaint, lab/LIMS systems, and other CMMS/ERP platforms.
• Navigate customer IT environments to establish connectivity — VPNs, service accounts, firewall rules, read-only database access — often with limited or no documentation.
• Reverse-engineer undocumented or poorly documented source schemas to identify the right data for integration.
• Build and own the extraction layer: connectors, API calls, direct database queries, file-based ingestion from heterogeneous client environments.
• Handle the reality that every customer’s data is messy in a different way — inconsistent tag naming, mismatched equipment IDs, unmaintained asset hierarchies.
Data Transformation and Pipeline Development• Design, build, and maintain data pipelines that clean, transform, and load extracted data into our reliability platform.
• Develop integration architecture and blueprints tailored to each customer’s source system landscape.
• Implement data quality checks, reconciliation processes, and monitoring to ensure ongoing accuracy.
• Build and maintain master data mapping strategies — including change management processes as clients execute MOCs, add equipment, or decommission assets.
• Own pipeline monitoring, alerting, and uptime SLAs for all production data extraction and integration systems. These are live production pipelines serving customers — when extraction fails, you are responsible for detecting the failure, diagnosing the root cause, and restoring the data flow within SLA.
Client Communication and Technical Leadership• Serve as the primary technical point of contact with customer IT teams for all data access and connectivity matters.
• Respond to detailed technical inquiries from client IT leadership (architecture questions, data mapping strategies, security concerns) with clarity and confidence.
• Lead discovery sessions with customers to understand their source systems, data flows, and integration requirements.
• Create and maintain architecture documentation, integration runbooks, and data dictionaries for each client engagement.
• Provide technical guidance and mentorship to team members and drive knowledge sharing across the data engineering team.
• Manage integration project plans, timelines, and deliverables across multiple concurrent client engagements. Drive accountability on milestones, coordinate dependencies with client IT teams, and ensure integrations are completed on schedule.
Strategy and Team Building• Lead the enterprise data integration strategy and platform architecture across the organization.
• Provide new ideas and approaches to the CTO and enterprise architecture team on data acquisition and integration best practices.
• Drive recruitment to build and grow a high-performing data engineering team.
• Continuously evaluate and adopt emerging data technologies and practices.
How We Measure SuccessMeasureTarget (Meets Expectations)
Projects delivered on time 80%
Team utilization 80%
New data solutions delivered per year 3
Production pipeline uptime (extraction/integration) 99.5%
Required Qualifications
• Hands-on experience extracting data from at least two of: OSIsoft PI, SAP PM/EAM, Maximo, eMaint, or similar industrial/operational systems. This is non-negotiable.
• Direct experience working with external customer or client IT teams to negotiate and establish data access (firewall rules, VPN connectivity, service accounts, API credentials).
• SQL proficiency — specifically the ability to explore unfamiliar database schemas and write extraction queries with little or no documentation.
• Python for data extraction, transformation, and pipeline automation.
• Experience with cloud-based data integration (Azure Data Factory, Azure Functions, or comparable).
• Strong knowledge of data integration patterns, ETL/ELT, APIs, and messaging protocols (REST, SOAP, OPC).
• Demonstrated experience with enterprise database technologies and data modeling.
• Excellent communication skills — you’ll be the person answering detailed technical emails from client IT directors and leading discovery calls.
Experience in oil and gas, refining, chemicals, or heavy industry environments.
Preferred Qualifications• Familiarity with reliability engineering concepts (RBI, CMMS workflows, asset hierarchy management, inspection data).
• Experience with Cognite Data Fusion (CDF) or similar industrial data platforms.
• Knowledge of PI Web API, PI SDK, or AF SDK for historian data extraction.
• Experience with OPC-UA/DA protocols for real-time industrial data.
• Background in data governance and compliance measures.
• Understanding of microservices architecture and containerization (Docker, Kubernetes).
• Experience with DevOps tools and practices (Azure DevOps, CI/CD pipelines).
Tools and Technologies You’ll Use• Data integration: Azure Data Factory, Python, SQL, REST/SOAP APIs, OPC protocols.
• Industrial source systems: OSIsoft PI, SAP PM/EAM, Maximo, eMaint, lab/LIMS platforms.
• Cloud platforms: Azure (primary), AWS (client environments).
• Analytics and reporting: Power BI, Tableau.
• DevOps: Azure DevOps, Docker, Kubernetes.
A Day in the Life Might Look Like• Morning: Jump on a call with a refinery client’s IT team to troubleshoot why their PI historian connection dropped overnight. Identify that a firewall rule expired and work with their network team to restore it.
• Mid-morning: Review a new client’s SAP PM export — the equipment hierarchy doesn’t match what’s in their CMMS. Dig into both schemas to build the mapping table.
• Afternoon: Write a Python extractor to pull lab data from a client’s LIMS system via a REST API that’s documented in a 4-year-old PDF with half the endpoints deprecated.
• Late afternoon: Respond to a detailed email from a client’s IT director asking about your master data management strategy and how you’ll keep mappings in sync as they execute MOCs and add new equipment.
• End of day: Mentor a junior data engineer on how to approach a new client whose data is entirely in flat files exported from a system that was decommissioned two years ago.
Sound like you? If you’ve spent your career figuring out how to get data out of systems that don’t want to give it up, and you can communicate that process to a client’s IT leadership team as confidently as you can write the extraction code, we want to hear from you.