AWS & Databricks Platform Engineer

Dice is the leading career destination for tech experts at every stage of their careers. Our client, Capgemini America, Inc., is seeking the following. Apply via Dice today!

Position Title : AWS & Databricks Platform Engineer

Location : Charlotte, NC (Onsite/Hybrid)

Experience : 8 Years

Employee Type : Full Time with Benefits

Job Description

We are seeking a highly skilled AWS & Databricks Platform Engineer with strong experience in cloud infrastructure provisioning, security, automation, and data platform integration. The ideal candidate will possess deep knowledge of AWS services, Infrastructure-as-Code (IaC), serverless architecture, and Databricks workspace configuration, along with excellent communication and problemsolving abilities.

Technical Skills & Qualifications

• Strong proficiency in AWS services such as CloudFormation (CFT), EC2, S3, Lambda, RDS, AWS SDK and CLI, and Terraform.
• Handson experience with AWS IaC tools including AWS CloudFormation for automated cloud deployments.
• Solid understanding of serverless architecture and microservices, with practical experience designing scalable AWS solutions.
• Knowledge of AWS security best practices including IAM, VPC, encryption mechanisms, and network isolation.
• Experience applying data governance controls, audit logging, and compliance frameworks.
• Strong analytical and troubleshooting skills for debugging and optimizing AWS resource utilization.
• Effective communication skills for collaborating with technical and nontechnical stakeholders.
  
  

Key Responsibilities

• Set up Databricks workspaces in designated AWS regions.
• Configure networking components including VPCs, subnets, security groups, and secure connectivity.
• Implement storage and credential setups, such as KMS encryption keys for data security.
• Remediate WIZ security violations.
• Define and implement Role-Based Access Controls (RBAC) using Unity Catalog.
• Integrate Databricks with Client s Identity Provider (SSO) and manage authentication flows.
• Ensure compliance with enterprise security standards and governance controls.
• Validate external data access strategies and configure connectors for S3 and other data sources.
• Enable ingestion frameworks and orchestrate notebooks for endtoend data pipelines.
• Support CI/CD deployment pipelines for Databricks notebooks and jobs.
• Implement monitoring for cluster lifecycle management, cost optimization, and performance tuning.
• Troubleshoot platform issues and manage error resolution during execution.
• Apply data governance and ML/GenAI governance controls per enterprise policies.
• Ensure proper audit logging and lineage tracking for all data assets.
• Collaborate with internal teams including EADP, CTO, ESF to align with architectural strategy and organizational best practices.
• Document configurations, processes, architectural decisions, and operational runbooks for future reference.