|
Description
|
-
Architect and manage a secure, scalable, and highly available Databricks Lakehouse Platform on AWS.
-
Design data platforms leveraging AWS S3, Delta Lake, Unity Catalog, and Databricks Workspaces for enterprise analytics and AI.
-
Define networking architecture including VPC, PrivateLink, IAM roles, security groups, and encryption standards.
-
Establish multi-environment strategies (Dev, Test, UAT, Prod) with automated provisioning through Terraform and Infrastructure as Code (IaC).
-
Implement enterprise-wide data governance, lineage, metadata management, and compliance controls using Unity Catalog.
-
Design high-performance data ingestion and processing architectures supporting batch, streaming, and real-time analytics workloads.
-
Enable AI and GenAI capabilities through Mosaic AI, Vector Search, Model Serving, AI/BI Dashboards, and Genie Spaces.
-
Define CI/CD, monitoring, logging, and observability frameworks using GitHub, Jenkins, CloudWatch, and Databricks Workflows.
-
Optimize platform performance, reliability, scalability, and cloud costs through FinOps and workload optimization practices.
-
Provide architectural leadership and best practices for Lakehouse modernization, advanced analytics, AI governance, and enterprise data platform adoption.
-
Core Technologies: Databricks, AWS S3, Delta Lake, Unity Catalog, Spark/PySpark, Terraform, GitHub, CI/CD, CloudWatch, Mosaic AI, Genie, Vector Search, MLflow, Kafka, Airflow, Kubernetes.
Certification :
Deliverables:
-Process Flows
-Mentor and Knowledge transfer to client project team members
-Participate as primary, co and/or contributing author on any and all project deliverables associated with their assigned areas of responsibility
-Participate in data conversion and data maintenance
-Provide best practice and industry specific solutions
-Advise on and provide alternative (out of the box) solutions
-Provide thought leadership as well as hands on technical configuration/development as needed.
-Participate as a team member of the functional team
-Perform other duties as assigned.
|
|
Top Skills
|
|
S.No
|
Skill Name
|
Experience
|
|
1
|
Databricks on AWS Platform Engineer – Platform Architecture
|
|
|
2
|
Architect and manage a secure, scalable, and highly available Databricks Lakehouse Platform on AWS.
|
|
|
3
|
Design data platforms leveraging AWS S3, Delta Lake, Unity Catalog, and Databricks Workspaces for enterprise analytics and AI.
|
|
|
4
|
Define networking architecture including VPC, PrivateLink, IAM roles, security groups, and encryption standards.
|
|
|
5
|
Establish multi-environment strategies (Dev, Test, UAT, Prod) with automated provisioning through Terraform and Infrastructure as Code (IaC).
|
|
|
6
|
Implement enterprise-wide data governance, lineage, metadata management, and compliance controls using Unity Catalog.
|
|
|
7
|
Design high-performance data ingestion and processing architectures supporting batch, streaming, and real-time analytics workloads.
|
|
|
8
|
Enable AI and GenAI capabilities through Mosaic AI, Vector Search, Model Serving, AI/BI Dashboards, and Genie Spaces.
|
|
|
9
|
Define CI/CD, monitoring, logging, and observability frameworks using GitHub, Jenkins, CloudWatch, and Databricks Workflows.
|
|
|
10
|
Optimize platform performance, reliability, scalability, and cloud costs through FinOps and workload optimization practices.
|
|
|
11
|
Provide architectural leadership and best practices for Lakehouse modernization, advanced analytics, AI governance, and enterprise data platform adoption.
|
|
|
