What are the responsibilities and job description for the Innovation and Automation Data Engineer (WDP) position at Bespoke Technologies, Inc.?
BT-161 - Innovation and Automation Data Engineer Lead (WDP)
Location: Chantilly/Herndon
As the Innovation and Automation Data Engineer Lead, you will be a hands-on leader of a specialized team that serves as the engineering backbone for our entire data organization. You will combine deep expertise in data engineering with a passion for automation to build the foundational frameworks that our Data Engineering and Data Product teams use every day. Your primary mission is to design, create, and manage reusable data pipeline templates, automated deployment patterns, and data quality frameworks. This is a "player-coach" role where you will set the technical vision for data engineering automation while mentoring a team and contributing directly to its most complex challenges.
Responsibilities
Location: Chantilly/Herndon
- MUST HAVE AN ACTIVE TS OR TS/SCI CLEARANCE TO APPLY. Those without an active security clearance will not be considered.**
As the Innovation and Automation Data Engineer Lead, you will be a hands-on leader of a specialized team that serves as the engineering backbone for our entire data organization. You will combine deep expertise in data engineering with a passion for automation to build the foundational frameworks that our Data Engineering and Data Product teams use every day. Your primary mission is to design, create, and manage reusable data pipeline templates, automated deployment patterns, and data quality frameworks. This is a "player-coach" role where you will set the technical vision for data engineering automation while mentoring a team and contributing directly to its most complex challenges.
Responsibilities
- Lead and mentor a team of automation-focused data engineers responsible for building and maintaining core data infrastructure frameworks.
- Design and develop standardized, reusable ETL/ELT data pipeline templates for ingestion, transformation, and data product creation.
- Architect and implement the CI/CD for Data (DataOps) strategy, enabling automated testing, validation, and deployment of data pipelines.
- Integrate and manage automated data quality, testing, and monitoring tools (e.g., Great Expectations) directly into the core pipeline templates.
- Develop and maintain a library of Infrastructure-as-Code (IaC) modules (e.g., Terraform) for provisioning common data engineering resources.
- Collaborate with platform and data engineering teams to identify complex problems, troubleshoot bottlenecks, and engineer automated solutions.
- Drive innovation by researching, prototyping, and implementing new technologies and patterns to improve the efficiency and reliability of our data engineering processes.
- Document data workflows and automation processes to ensure they are understood and adopted across the organization.
- 5 years of experience in data engineering, with a strong focus on automation and platform development.
- At least 2 years of experience in a technical leadership or mentorship role, supervising a team or guiding junior engineers.
- Expert-level skills in Python and SQL.
- Proven experience building and optimizing data pipelines on platforms like Databricks or Snowflake.
- Hands-on expertise with Infrastructure-as-Code (Terraform) and CI/CD tools (e.g., GitLab CI, Jenkins).
- Deep understanding of data architecture, data modeling, and modern data warehousing concepts.
- Direct experience with the data platform.
- Experience with data orchestration tools like Airflow.
- Familiarity with containerization technologies (Docker, Kubernetes).
- Strong project management skills with experience in cross-functional initiatives.