What are the responsibilities and job description for the AWS Databricks Platform Administrator position at Argyllinfotech?
- Job Title: AWS Databricks Platform Administrator
- End Customer: LTI Mindtree
- Location: Raritan, NJ (On-site / Local requirement)
- Position Type: Contract (6 Months)
Key Responsibilities
- Platform Administration: Own the administration, security, optimization, and support of Databricks accounts, workspaces, Unity Catalog metastores, clusters, SQL warehouses, jobs, instance pools, repositories, libraries, secrets, and access controls on AWS.
- Architecture & Governance: Design and maintain secure, scalable, and highly available Databricks platform architectures aligned with enterprise cloud standards. Implement robust data governance utilizing Unity Catalog (catalogs, schemas, external locations, storage credentials, privileges, lineage, and auditability).
- AWS & Infrastructure Integration: Configure and troubleshoot core AWS services, including S3, IAM, VPC, PrivateLink, KMS, CloudWatch, CloudTrail, Route 53, and Secrets Manager.
- FinOps & Cost Management: Drive FinOps practices by analyzing DBU and AWS consumption, enforcing tagging strategies, establishing budgets, optimizing compute configurations, and reducing idle or inefficient resources.
- Automation & CI/CD: Automate workspace provisioning, configuration, policy enforcement, and operational tasks utilizing Terraform, Databricks Asset Bundles, Databricks CLI/SDK, Python, and CI/CD pipelines.
- Operations & Incident Management: Lead incident, problem, change, release, vulnerability, and service-request management while meeting established SLAs and operational KPIs. Plan and execute runtime upgrades, disaster recovery tests, and backup validations.
- Collaboration: Partner closely with data engineering, analytics, security, networking, and cloud engineering teams to onboard workloads and resolve complex technical issues.
Required Experience & Qualifications
- Overall Experience: 9 years of overall IT experience in cloud infrastructure, data platforms, DevOps, platform engineering, or production administration.
- Databricks Expertise: 5 years of hands-on Databricks administration experience with enterprise deployments on AWS.
- Core Technical Competencies: Strong command of Databricks account console workspace administration, Unity Catalog, Delta Lake, SQL warehouses, cluster policies, and system audit logs.
- AWS Competencies: Deep knowledge of AWS networking, security, storage encryption, monitoring, and identity services (VPC, S3, IAM, KMS, PrivateLink, CloudWatch, CloudTrail).
- Automation & Scripting: Hands-on experience with Infrastructure as Code (Terraform), Python, shell scripting, REST APIs, and CI/CD tooling.
- Foundational Knowledge: Solid working knowledge of Linux administration, networking, DNS, firewalls, certificates, proxies, and secure connectivity patterns, alongside a good understanding of Apache Spark architecture and distributed data workloads.
- Education: Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related discipline (or equivalent practical experience).
Mandatory Skills
- AWS S3
- Databricks
- Redshift
Important Submission Guidelines & Notes
- Work Authorization: Candidate must be authorized to work in the US without sponsorship. The expiration date must match the uploaded work authorization document.
- Screening: Candidates are subject to a background check and drug screen. All candidates selected for an interview must complete a mandatory identity verification process.