What are the responsibilities and job description for the Databricks Architect position at AptoNet Inc?
Location: Washington, DC (onsite)
Citizenship: U.S. citizenship required
Background Requirements: Must possess or be eligible to successfully complete a Public Trust background investigation.
About the job
We are seeking a Databricks Administrator / Subject Matter Expert to lead the design, governance, optimization, and adoption of the Databricks Lakehouse platform across the organization. This role will serve as the primary technical authority for Databricks, supporting data engineering, analytics, machine learning, and AI initiatives while driving platform best practices, performance optimization, security, and scalability.
Responsibilities
- Lead the design, implementation, and optimization of enterprise Databricks solutions.
- Manage and support key Databricks capabilities including Unity Catalog, Delta Lake, Databricks SQL, Workflows, Jobs, MLflow, and Mosaic AI.
- Design and maintain scalable data ingestion, transformation, and orchestration pipelines.
- Optimize Spark workloads, SQL performance, cluster utilization, and overall platform efficiency.
- Establish and enforce governance, security, compliance, and data management standards.
- Support machine learning and Generative AI initiatives utilizing MLflow, Vector Search, Feature Store, and LLM capabilities.
- Serve as the escalation point for complex Databricks production issues and performance challenges.
- Develop integrations between Databricks, cloud platforms, orchestration tools, BI solutions, and enterprise applications.
- Implement monitoring, logging, alerting, and operational dashboards for platform visibility and reliability.
- Collaborate with Data Engineering, Data Science, Analytics, Infrastructure, and business stakeholders to deliver data and AI solutions.
- Provide technical leadership, mentoring, training, and platform best practices across the organization.
Required Skills
- Bachelor's degree in Computer Science, Information Technology, Business, or a related field, or equivalent experience.
- U.S. Citizenship required.
- 7 years of experience in Data Engineering, Data Architecture, or related data platform roles.
- 4 years of dedicated hands-on Databricks experience.
- Proven experience implementing Unity Catalog, including cross-workspace governance and fine-grained security controls.
- Deep expertise with Apache Spark performance tuning and optimization.
- Experience supporting production Databricks environments at enterprise scale.
- Hands-on experience building Generative AI, LLM, or machine learning solutions using Databricks technologies.
- Advanced proficiency in Python, Scala, and SQL.
- Experience deploying and managing Databricks infrastructure using Terraform or other Infrastructure-as-Code tools.
- Experience implementing secure Databricks networking architectures, including Private Link, VPC Peering, encryption, and access controls.
- Strong experience with CI/CD pipelines using GitHub Actions, Azure DevOps, GitLab, Databricks CLI, or Databricks Asset Bundles.
- Strong troubleshooting, architecture, communication, and stakeholder management skills.
Preferred Skills
- Databricks Certified Solutions Architect.
- Databricks Certified Data Engineer Professional.
- Databricks Certified Machine Learning Professional.
- AWS Certified Solutions Architect or Azure Solutions Architect Expert certification.
- Experience with Mosaic AI, Feature Store, Vector Search, and MLflow.
- Experience designing enterprise-scale data platforms, AI ecosystems, and modern Lakehouse architectures.
- Experience supporting highly regulated environments with strong governance and compliance requirements.