Data Engineer is responsible for building a leading-edge Data & Analytics platform for enabling our new Troon Data Estate architecture & related analytics, business domain specific analytics & management (Finance, HR, Operations, & other), and general enterprise analytics needs. Designs, develops, maintains, and supports the cloud-based (Microsoft Azure) big data platform and uses modern data engineering design patterns and tools across the MSFT Data Platform.

$100,000-$130,000 BOE

Essential Duties:

• Designs, builds and maintains scalable, automated data pipelines to enable Reporting, Data Visualization, Advanced Analytics, Data Science, and Machine Learning solutions.
• Supports critical data pipelines with a scalable distributed architecture, including data ingestion (streaming, events, and batch), data integration (ETL, ELT, Azure Data Factory), and distributed data processing using Databricks Data/or other & Analytics and Azure Cloud Technology Stacks.
• Builds cloud data solutions using multiple technologies, such as SQL, Python, Data Lake (Databricks Delta Lake/or other), Cloud Data Warehouse (Azure Synapse), RDBMS, NoSQL databases.
• Implements best practices for data management in Microsoft Azure, such as data quality, data governance, data security, data lineage, data cataloging, etc.
• Deploys, automates, maintains, and manages cloud-based production systems to ensure the availability, performance, scalability, and security of production systems.
• Works with other engineers and stakeholders to understand business requirements and translate them into data models and architectures that meet performance, scalability, reliability, and security standards.
• Owns end-to-end design and development, testing, the release of critical components using Databricks technology stack and Microsoft Azure cloud platforms and services.

Education/Experience:

• Minimum BA or BS degree in Computer Science, Information Systems, or related field required. MS in Business Analytics or related discipline preferred.
• Minimum 6 years of experience in creating robust enterprise-grade data engineering pipelines using SQL, Python, Apache Spark, ETL, ELT, Databricks Technology Stack, Azure Cloud Services, Cloud-based Data and Analytics platforms required. 4-5 years preferred.
• Experience with Microsoft Azure Data Services and SQL Server, tasks to include design, implementation, maintenance and performance monitoring of database instances.
• Experience in distributed data (structured, semi-structured, unstructured, streaming) processing techniques using Apache Spark, Hadoop, Hive, Kafka, and big data ecosystem technologies preferred.

Job Knowledge, Skill, and Ability Preferences

• Knowledge and understanding of Microsoft Entra and RBAC models for data security and access controls a plus.
• Emphasis on Azure SQL virtual machines (IasS) as well as Azure SQL Managed instances (PaaS), with a primary goal of providing consistent database availability and performance for all applications utilizing these environments.