What are the responsibilities and job description for the Databricks Platform Manager/Engineer position at Prama Innovations India Pvt. Ltd.?
Job Title: Databricks Platform Manager/Engineer
Location: NY (Hybrid Onsite)
Type: Contract
Job Summary
We are looking for a Databricks Platform Manager/Engineer with strong hands-on experience administering enterprise Databricks environments on AWS. The role focuses on platform administration, workspace and cluster management, Unity Catalog, AWS IAM, security/governance, Terraform/IaC, CI/CD, monitoring, and cost optimization. Ideal candidates will have 3–5 years of production Databricks platform administration experience and a strong AWS/cloud infrastructure background.
Key Responsibilities
- Administer and support Databricks workspaces across Dev, Test, UAT, and Production.
- Manage clusters, cluster policies, instance pools, job clusters, and Databricks Runtime upgrades.
- Administer Unity Catalog, catalogs, schemas, permissions, and RBAC.
- Manage AWS IAM integration, SSO, service principals, secrets, and access controls.
- Implement and support Terraform/IaC and CI/CD for Databricks platform deployments.
- Manage Git/Databricks Repos and automated deployment processes.
- Monitor platform health, performance, jobs, clusters, and production environments.
- Troubleshoot platform issues, incidents, and performance problems.
- Support audit logging, security, governance, and compliance requirements.
- Identify opportunities for Databricks and AWS cost optimization.
- Partner with Data Engineering, DevOps, Cloud, Security, and Infrastructure teams.
Required Skills
- Strong hands-on Databricks Platform Administration
- Databricks Workspace Management
- Cluster Management & Cluster Policies
- Unity Catalog
- Databricks Runtime
- Service Principals, Secrets & PATs
- AWS IAM
- AWS S3, VPC, EC2, KMS, Security Groups
- Terraform / Infrastructure as Code
- Git and CI/CD
- Azure DevOps, GitHub Actions, or Jenkins
- Platform Monitoring & Troubleshooting
- Production Support
- Cost Optimization
Preferred
- Experience supporting enterprise Databricks environments with 100 users
- Experience with AWS PrivateLink / VPC Endpoints
- AWS Organizations / Control Tower
- CloudWatch, Datadog, or Splunk
- Delta Lake / Lakehouse architecture
- Exposure to Spark internals
- Experience supporting Databricks AI/ML workloads
- Databricks or AWS certifications
Salary : $50 - $55