What are the responsibilities and job description for the Data Engineer position at Bespoke Technologies, Inc.?
BT-153 – Data Engineer (Associate Manager)
Location: Chantilly/Herndon
As a senior Data Engineer, you will be a key technical contributor on the team building and maintaining the critical data pipelines for a data platform. You will apply your deep expertise to design, develop, and optimize robust ETL/ELT processes that move and transform data. In this role, you will not only build solutions but also mentor junior engineers and help drive the adoption of modern data engineering practices and automation.
Responsibilities
Location: Chantilly/Herndon
- MUST HAVE AN ACTIVE TS OR TS/SCI CLEARANCE TO APPLY. Those without an active security clearance will not be considered.**
As a senior Data Engineer, you will be a key technical contributor on the team building and maintaining the critical data pipelines for a data platform. You will apply your deep expertise to design, develop, and optimize robust ETL/ELT processes that move and transform data. In this role, you will not only build solutions but also mentor junior engineers and help drive the adoption of modern data engineering practices and automation.
Responsibilities
- Design, build, and maintain scalable and resilient data pipelines for both new data sources and existing systems within the data ecosystem.
- Implement and optimize pipelines using modular patterns, such as Databricks Delta Live Tables, following governance from the architecture team.
- Develop and integrate automated monitoring, alerting, and data quality checks into every pipeline to reduce downtime and ensure data integrity.
- Utilize Infrastructure-as-Code (IaC) with tools like Terraform to create consistent, repeatable deployments and CI/CD processes.
- Analyze complex source data formats and collaborate with data scientists and stakeholders to design transformations that meet objectives.
- Participate in and lead code reviews, enforce best practices, and mentor junior data engineers.
- Troubleshoot and resolve complex issues with data pipelines, and contribute to the ongoing improvement of O&M processes.
- 6 years of experience in data engineering.
- Strong experience with the development and maintenance of extract, transform, and load (ETL) tools and services.
- Proficiency in Python, SQL, Spark, and PySpark.
- Hands-on experience with cloud environments (AWS, Azure) and data platforms like Databricks, Palantir, or Snowflake.
- Experience working in an Agile/Scrum development environment.
- Familiarity with data orchestration and data quality processes.
- Active Top Secret/SCI security clearance.
- Direct experience engineering data pipelines for the data platform.
- Experience working in high-security environments (e.g., SIPR, JWICS).
- Hands-on experience with Databricks Delta Live Tables and Terraform.
- Experience with containerization technologies like Docker or Kubernetes.
- Knowledge of COTS and open-source data engineering tools such as NiFi or ElasticSearch.