What are the responsibilities and job description for the Senior Databricks Data Engineer (AWS) - Remote Opportunity position at Booker DiMaio?
Previous Experience With a Government Client Is Required.
This is a remote position with a federal client who is based in Lanham, MD. Candidates must reside and perform work within the United States.
Position Overview
We are seeking two Senior Databricks Data Engineers to support a large-scale federal Enterprise Data Platform (EDP) modernization initiative. This role will focus on designing, developing, optimizing, and governing cloud-native data solutions utilizing Databricks Lakehouse technologies and Amazon Web Services (AWS).
The ideal candidate possesses extensive experience building enterprise-scale data pipelines, implementing data governance controls, optimizing large-scale data processing workloads, and enabling secure analytics capabilities in cloud environments. This position offers the opportunity to support a mission-critical federal data platform supporting trusted data intake, governed access, and advanced analytics capabilities.
Key Responsibilities
This is a remote position with a federal client who is based in Lanham, MD. Candidates must reside and perform work within the United States.
Position Overview
We are seeking two Senior Databricks Data Engineers to support a large-scale federal Enterprise Data Platform (EDP) modernization initiative. This role will focus on designing, developing, optimizing, and governing cloud-native data solutions utilizing Databricks Lakehouse technologies and Amazon Web Services (AWS).
The ideal candidate possesses extensive experience building enterprise-scale data pipelines, implementing data governance controls, optimizing large-scale data processing workloads, and enabling secure analytics capabilities in cloud environments. This position offers the opportunity to support a mission-critical federal data platform supporting trusted data intake, governed access, and advanced analytics capabilities.
Key Responsibilities
- Design, develop, and maintain scalable data ingestion, transformation, and publishing pipelines utilizing Databricks and AWS services.
- Implement and optimize Databricks Lakehouse capabilities including Unity Catalog, Delta Live Tables, Auto Loader, Databricks SQL, and Delta Sharing.
- Build and maintain governed data products supporting operational, analytical, reporting, and machine learning workloads.
- Develop and support medallion architecture data pipelines and enterprise data quality frameworks.
- Implement data governance controls, metadata management, lineage tracking, and data retention policies.
- Collaborate with cloud engineers, architects, cybersecurity specialists, and business stakeholders to deliver secure, production-ready solutions.
- Optimize platform performance through partitioning, clustering, caching, workload tuning, and query optimization techniques.
- Support analytics enablement through semantic layers, dashboards, reporting solutions, and self-service data access capabilities.
- Participate in architecture reviews, operational readiness activities, platform modernization initiatives, and continuous improvement efforts.
- Create and maintain technical documentation, design artifacts, operational procedures, and engineering standards.
- Bachelor's degree in Computer Science, Information Systems, Engineering, Mathematics, or a related discipline.
- Minimum eight (8) years of experience in cloud-based data engineering, platform development, or data architecture.
- Minimum three (3) years of hands-on Databricks experience in production environments.
- Experience designing, developing, and supporting large-scale ETL/ELT solutions.
- Strong experience with AWS services including S3, IAM, Glue, Athena, Lambda, Redshift, and CloudWatch.
- Experience implementing enterprise data governance and security controls.
- Strong SQL and Python development skills.
- Experience working with data lakes, data warehouses, and cloud-native analytics platforms.
- Strong troubleshooting, performance tuning, and optimization experience.
- Databricks Certified Data Engineer Associate or Professional certification.
- AWS Data Analytics Specialty certification.
- Experience supporting federal government or highly regulated environments.
- Experience with real-time streaming, CDC solutions, and event-driven architectures.
- Familiarity with FedRAMP, FISMA, and NIST security frameworks.
- Experience supporting large-scale enterprise data platforms.