Databricks Data Engineer

Software Engineering III – Databricks Data Engineer (Remote | U.S. Citizenship Required)

Remote – quarterly travel to Gaithersburg, MD

We are seeking a highly skilled Software Engineering III – Databricks Data Engineer to join our growing team and play a key role in building scalable, secure, and high-performing data solutions within an enterprise Microsoft Azure cloud environment.

This role is ideal for a hands-on Data Engineer with strong expertise in Databricks, Apache Spark, Azure data services, and modern data lake architecture. You will be responsible for designing, developing, and optimizing enterprise-grade data pipelines, supporting advanced analytics and AI-driven workflows, and collaborating with both technical and business stakeholders to deliver impactful data solutions.

If you enjoy solving complex data engineering challenges, working with large-scale datasets, and contributing to mission-critical cloud analytics platforms, this is a great opportunity to make a meaningful impact.

Key Responsibilitie

• sDesign, develop, and optimize scalable data pipelines and analytics solutions using Databricks and Medallion Architectur
• eBuild and maintain data ingestion pipelines for structured, streaming, and unstructured dat
• aManage and support ingestion routines for multi-terabyte datasets across multiple Databricks workspace
• sTranslate business and technical requirements into robust Azure-based data engineering solution
• sIntegrate data from multiple enterprise sources to enable high-quality analytics, reporting, and insight
• sMaintain and enhance ETL/ELT processes with a strong focus on data quality, reliability, and performanc
• eMonitor platform health, troubleshoot production issues, and optimize Spark workloads, cluster performance, and platform stabilit
• yImplement data management strategies to ensure data integrity, availability, accessibility, and governanc
• eIdentify and drive cost optimization opportunities across data storage, compute, and analytics workload
• sIntegrate and work with key Azure services, including
• :Azure Function
• sAzure Storage Service
• sAzure Data Factor
• yAzure Log Analytic
• sAzure User Management / Access Control
• sProvision and manage cloud infrastructure using Infrastructure-as-Code (IaC
• )Apply best practices for data security, compliance, governance, and federal/Public Trust standard
• sPartner with cross-functional stakeholders, including business users, architects, analysts, and engineering team
• sSupport AI/ML and advanced analytics workflows by enabling reliable and scalable data foundation
• sProvide client-facing support, technical consultation, and clear solution documentatio
• nCreate and maintain architecture diagrams, interface diagrams, and technical documentatio
• nMentor and review the work of junior technical team members as neede

d
Required Technical Skil

• lsStrong hands-on experience with Databricks platform development, administration, and maintenan
• ceExpertise in Apache Spa
• rkProficiency in Python and Spark (R experience is a plu
• s)Strong experience with Azure cloud services, includin
• g:Azure Functio
• nsAzure Storage Servic
• esAzure Data Facto
• ryAzure Log Analyti
• csUser / Identity Manageme
• ntExperience building data pipelines for structured, streaming, and unstructured da
• taStrong understanding of ETL/ELT development, orchestration, and data quality manageme
• ntDeep knowledge of Medallion Architecture and modern data lake / lakehouse design patter
• nsExperience with Infrastructure-as-Code (Ia
• C)Knowledge of data governance, includin
• g:Metadata manageme
• ntEnterprise data catal
• ogData quali
• tyData security / access contro
• lsExperience with CI/CD automation for data engineering workflo
• wsFamiliarity with Agile / Scrum methodologi
• esExposure to cloud platforms such as Azure (required) and AWS (preferre

d)
Preferred / Nice-to-Have Ski

• llsExperience with .NET developm
• entAzure cloud certificati
• onsUnderstanding of FinOps / cloud cost management princip
• lesExposure to AI/ML platforms, advanced analytics, or modern AI technolog

ies
Qualifications & Experi

• enceBachelor’s degree in Computer Science, Engineering, or related field with 3 years of relevant experie
• nc
• e,ORMaster’s degree in a related field with 2 years of relevant experi
• enceDatabricks Data Engineer Certification strongly preferred / requ
• ired2 years of hands-on experience supporting and maintaining the Databricks platform and Spark-based develop
• ment3 years of experience designing and developing cloud-based data ingestion and processing pipel
• inesProven experience working directly with clients / end users and providing front-line technical sup
• portStrong ability to create and maintain technical architecture and interface documenta

tion
Work Authorization & Clearance Require

• mentsU.S. Citizenship is req
• uiredMust be eligible to obtain or currently hold a Public Trust clea

rance
Work Location /

• TravelRemote po
• sitionMust be comfortable with quarterly travel to Gaithersbu
• rg, MDCandidates located in the Greater DC / Maryland / Virginia area are highly pre