What are the responsibilities and job description for the Senior Data Engineer position at Haystack?
A world-leading tax compliance software company, trusted by industry leaders, is seeking a talented individual to join their innovative team. This company provides a robust platform, powered by an advanced data engine, designed to reduce redundancy and risk while managing complex tax compliance implications for organizations.
The Role
The Role
- Lead the design and implementation of enterprise-grade data pipelines on Snowflake and Azure, covering ingestion, transformation, and serving layers for governed data products.
- Architect ingestion and integration patterns across various source systems, applying CDC, batch, and streaming patterns as appropriate.
- Establish and maintain transformation frameworks using dbt, defining layering and naming conventions, and ensuring all pipelines are version-controlled, tested, and observable.
- Drive schema governance and data modeling discipline, including conceptual, logical, and physical data models, naming conventions, and managing breaking changes.
- Implement and maintain data lineage, catalog, and metadata management to ensure traceable sources, definitions, owners, and certification status for KPIs.
- Partner with leadership on data platform strategy, Center of Excellence operating model, intake processes, prioritization, and technical roadmap.
- Bachelor's degree in Computer Science, Computer Engineering, Information Systems, or equivalent demonstrated experience.
- 4-7 years of professional data engineering experience with demonstrated technical leadership on enterprise-grade data platforms.
- Advanced proficiency in SQL for complex query authoring, performance tuning, and schema design at scale.
- Hands-on proficiency with modern data transformation tooling (dbt, Dataform, or equivalent) and workflow orchestration tooling (Airflow, Azure Data Factory, or equivalent).
- Production experience with Snowflake, including warehouse design, role-based access control, performance optimization, and cost management.
- Strong working experience with Microsoft Azure data services (Azure Data Factory, Azure SQL, Storage, Key Vault) and integration with Azure security and identity services.
- Opportunity to operate within an AI-first development culture, leveraging cutting-edge AI development assistants.
- Mentorship opportunities for Data Engineers, contributing to team growth and technical capability.
- A role central to shaping the data foundation for trusted decision-making and AI-powered capabilities.
- Emphasis on continuous professional development, particularly in AI-assisted development and emerging data platform capabilities.