What are the responsibilities and job description for the Azure Databricks Architect position at Jobs via Dice?
Dice is the leading career destination for tech experts at every stage of their careers. Our client, Rapsys Technologies, is seeking the following. Apply via Dice today!
Azure DataBricks Solution Architect
> We are looking for an experienced Databricks Architect with extensive knowledge in data engineering, cloud platforms, and advanced analytics. This role is pivotal in designing, implementing, and optimizing scalable data solutions on the Databricks platform.
As a key technical leader, you will work closely with engineering, data science, and business teams to create secure, high-performance, and cost-efficient data pipelines and AI-driven systems.
**Key Responsibilities**
Azure DataBricks Solution Architect
> We are looking for an experienced Databricks Architect with extensive knowledge in data engineering, cloud platforms, and advanced analytics. This role is pivotal in designing, implementing, and optimizing scalable data solutions on the Databricks platform.
As a key technical leader, you will work closely with engineering, data science, and business teams to create secure, high-performance, and cost-efficient data pipelines and AI-driven systems.
**Key Responsibilities**
- Design and implement scalable data platforms and pipelines using Databricks (including Spark, Delta Lake, MLflow, etc.).
- Work with stakeholders to develop modern data architectures that meet business requirements.
- Lead the migration from legacy platforms to the Databricks Lakehouse architecture.
- Establish data governance, quality, and security standards in collaboration with InfoSec and Compliance teams.
- Optimize data ingestion, transformation, and consumption processes.
- Mentor data engineers and developers on best practices for using Databricks.
- Integrate Databricks with enterprise tools such as Power BI, Tableau, Kafka, Snowflake, and others.
- Develop and implement machine learning/AI workflows within a Databricks environment.
- Collaborate with cloud architects to ensure a robust infrastructure setup on AWS, Azure, or Google Cloud Platform.
- Produce technical documentation, architectural diagrams, and data models.
- Over 8 years of experience in data engineering or architectural roles.
- More than 5 years of experience in architecting solutions with Databricks.
- Expertise in Apache Spark, Delta Lake, MLflow, and Lakehouse architecture.
- Proficiency in SQL, Python, and/or Scala.
- Practical experience with cloud platforms such as AWS, Azure, or Google Cloud Platform.
- Strong understanding of distributed computing, performance tuning, and data security.
- Experience with CI/CD pipelines, DevOps tools, and orchestration tools like Airflow or Data Factory.
- Databricks Certified Data Engineer or Solutions Architect.
- Knowledge of MLOps and deploying machine learning models into production.
- Experience with data mesh, data fabric, or similar enterprise data architectures.