What are the responsibilities and job description for the Databricks Solution Architect position at SumasEdge Corporation?
Job Description
- Design and implement scalable data solutions using Databricks on Azure or AWS.
- Build and optimize ETL/ELT pipelines for large-scale data processing.
- Develop data lake and data warehouse architectures.
- Work with Spark, PySpark, SQL, and Delta Lake for data engineering solutions.
- Collaborate with business and technical teams to gather requirements and design solutions.
- Ensure data security, governance, and performance best practices.
- Integrate Databricks with cloud services such as Azure Data Factory, AWS S3, or ADLS.
- Troubleshoot performance issues and optimize data workflows.
- Provide technical guidance and support to development teams.
- Experience with CI/CD, Git, and cloud platforms (Azure/AWS) is preferred.