Databricks Architect – AWS Cloud

Databricks Architect AWS Cloud

Location: Minnesota (Local candidates only)

Duration: Long-term Contract

Job Summary

We are seeking a highly experienced Databricks Architect with strong AWS expertise to lead the architecture, design, and optimization of large-scale data engineering and analytics solutions. The ideal candidate will have hands-on experience building enterprise-grade data platforms using Databricks, Delta Lake, and AWS cloud services.

Key Responsibilities

• Lead end-to-end architecture of Databricks-based data and analytics platforms on AWS.
• Design scalable ELT/ETL pipelines using Databricks, Spark, and Delta Lake.
• Architect data ingestion, data quality, governance, and lineage frameworks.
• Implement CI/CD pipelines for Databricks notebooks and jobs.
• Optimize Spark workloads and cluster configurations for performance and cost.
• Integrate Databricks with AWS services (S3, Glue, Lambda, Redshift, EMR, Kinesis, IAM).
• Provide architectural guidance and governance across engineering teams.
• Collaborate with product owners, data engineers, and business stakeholders.
  
  

Required Skills

• 10 years in data engineering/architecture; 4 years in Databricks.
• Strong hands-on Spark (PySpark/Scala) experience.
• Expert knowledge of AWS cloud architecture-S3, IAM, Glue, Lambda, Step Functions, EMR, Redshift.
• Experience implementing Delta Lakehouse architecture.
• Strong knowledge of CI/CD (Azure DevOps, GitHub Actions, Jenkins, etc.).
• Excellent communication and stakeholder management skills.