What are the responsibilities and job description for the Principal Data Engineer position at BrickRed Systems?
We are seeking an experienced Principal Data Engineer with expertise in designing and leading enterprise-scale Data Engineering solutions. The ideal candidate should have strong hands-on experience in Snowflake, Databricks, Python, PySpark, SQL, ETL/ELT pipelines, Azure Data Factory, Airflow, Kafka, dbt, Azure Cloud, and Data Governance. This role involves architecting scalable cloud-native data platforms, leading engineering teams, implementing CI/CD standards, and delivering high-performance data solutions for enterprise finance and analytics.
Required Skills
Primary Skills
- Snowflake
- Databricks
- Python / PySpark
- Advanced SQL
- ETL / ELT Pipeline Development
- Azure Data Factory (ADF)
- Airflow
- Kafka / Event Hub
- dbt
- Azure Cloud (ADLS Gen2, Key Vault)
- Data Modeling
- CI/CD
- Terraform
- Data Governance
Experience
- 10 years of Data Engineering experience.
- Strong expertise in Snowflake and Databricks platforms.
- Hands-on experience with Python, PySpark, SQL, ETL/ELT, and cloud-native data engineering.
- Experience designing enterprise-scale batch, streaming, and real-time data pipelines.
- Strong knowledge of Azure Cloud, Kafka, dbt, and orchestration frameworks.
- Experience leading engineering teams and enterprise data platform initiatives.
- FAANG / Product-Based / Tier-1 Company experience preferred.
Key Responsibilities
- Design and architect enterprise-scale ETL/ELT and streaming data pipelines.
- Lead implementation of Snowflake, Databricks, and cloud-native data platforms.
- Develop scalable Python/PySpark data processing frameworks.
- Build real-time data pipelines using Kafka, Event Hub, and Spark Structured Streaming.
- Establish engineering standards for CI/CD, testing, deployment, and automation.
- Implement enterprise data quality, governance, monitoring, and security frameworks.
- Optimize cloud infrastructure, storage, and compute performance.
- Design scalable enterprise data models and semantic layers.
- Mentor engineering teams and establish reusable engineering frameworks.
- Collaborate with architects, product managers, and business stakeholders to deliver enterprise data solutions.
Desired Competencies
- Strong analytical and problem-solving skills.
- Excellent leadership and mentoring abilities.
- Strong communication and stakeholder management skills.
- Experience architecting enterprise-scale data platforms.
- Deep understanding of cloud-native data engineering.
- Strong knowledge of finance and revenue data domains.
- Experience driving engineering best practices and platform modernization.
Key Deliverables
- Enterprise-scale Data Engineering solutions.
- ETL/ELT pipeline architecture and implementation.
- Snowflake and Databricks platform optimization.
- Streaming and real-time data pipelines.
- CI/CD implementation and automation.
- Data quality and governance frameworks.
- Cloud platform optimization.
- Engineering standards and reusable frameworks.
ABOUT BRICKRED SYSTEMS
BrickRed Systems is a global leader in next-generation technology consulting and workforce solutions, specializing in delivering high-quality talent across digital, engineering, marketing, analytics, finance, operations, and business transformation domains. With a strong emphasis on innovation, scalability, and client success, BrickRed Systems helps organizations solve complex business challenges by providing skilled professionals across strategy, technology, creative, and operational functions. BrickRed fosters a culture of continuous learning, collaboration, and excellence, enabling professionals to contribute to high-impact global initiatives while advancing their careers.