What are the responsibilities and job description for the Databricks Platform Architect (AWS) position at CCS Global Tech?
About the role
The client needs an enterprise-grade Databricks foundation on AWS: accounts, workspaces, networking, identity, and compute, built once and built right. You own the architecture and the hands-on build, and you prove it works with measured evidence, not assertions. You will also act as the technical reviewer for platform design decisions and for issues the delivery teams cannot resolve.
What you will do
1. Design the account and workspace topology, mapped to environments (dev/test/prod), business units, and data domains, with the isolation and governance model between them.
2. Design the Unity Catalog foundation: metastore strategy, catalog structure per environment, storage credentials, external locations, and audit through system tables.
3. Design network and workspace isolation: VPC, subnet strategy, PrivateLink, and transit/peering, preventing unintended BU-to-BU routing and public exposure.
4. Own compute strategy across classic and serverless: optimization, scalability, reliability, cost controls, and FinOps tagging and reporting.
5. Implement identity: SSO/MFA, SCIM provisioning, group-based access, and automated deprovisioning.
6. Establish service principal and workload identity: automated provisioning, least privilege, authentication, auditing, and revocation.
7. Design encryption and key management, including customer-managed keys.
8. Build the platform as code with the Databricks Terraform provider, and Databricks Asset Bundles for CI/CD, so environments are repeatable and auditable.
9. Run proof-of-control tests and quantify results: provisioning and revocation time, isolation, and network exposure.
10. Work with the client’s network, identity, and security teams, and support their architecture and security reviews through to approval.
11. Document the reference architecture, DR/HA approach, observability model, and handover runbook.
What you need
• 8 years in cloud architecture, 3 on Databricks on AWS.
• Hands-on experience standing up Databricks accounts and workspaces from scratch in enterprise environments, not only working within an existing setup.
• AWS at depth: VPC, PrivateLink, IAM, S3, and KMS.
• Unity Catalog design and administration.
• Identity provisioning: SSO/MFA, SCIM, group-based access, automated deprovisioning.
• Service principal lifecycle: provisioning, least privilege, authentication, revocation.
• Terraform, including the Databricks provider.
• Databricks compute architecture: classic and serverless, optimization, cost controls.
• Experience designing and running proof-of-control tests with quantified outcomes.
• Ability to brief engineers and executives, and to hold design positions in client review forums.
Nice to have
• Databricks Cloud Platform Architect accreditation or Platform Administrator certification.
• AWS Solutions Architect Professional or Security Specialty.
• Regulated-sector delivery experience.