What are the responsibilities and job description for the Hiring! Databricks Enterprise Architect - Minneapolis, MN position at PropelSys Technologies LLC.?
- Title - Databricks Architect
- Location Minneapolis, MN
- Work Mode Hybrid/ Onsite (the candidate shall be required to work from client's office for 2 - 3 days in a week)
Enterprise Architect:
Provide guidance and support to teams and analysts.
Expert on Databricks and AWS, with minimum 10 years of experience
Databrick certification is a must
Collaborate on data strategy with business and IT partners.
Identify and address data issues.
Lead architecture, design, and implementation of scalable and efficient data pipelines and
analytics solutions on Databricks.
Work closely with data engineering, platform, and analytics teams to integrate Databricks
with AWS S3 Data Lake, Redshift, Talend, SAP BW/HANA, Oracle, and other tools.
Architect solutions supporting batch and real-time data processing using Delta Lake,
Spark, and ML Flow.
Collaborate with business stakeholders to understand data requirements, assess current
state, and provide strategic direction for future-state architecture.
Ensure best practices for data quality, governance, and security are implemented.
Own design, development, and delivery of pipelines and models as per the project plan.
Ensure best practices are followed in performance, security, and governance.
Provide project documentation and conduct KT sessions
Ensure following acceptance criteria is met, by working along with current development
and pool partners
o Successful ingestion and transformation of in-scope objects.
o Validated and reconciled transformed data within defined thresholds.
o Fully operational Snowflake warehouse with business-validated datasets.
o Documented operational playbooks and KT completed.
o No critical issues post go-live during hyper-care window