What are the responsibilities and job description for the Solutions Architect 4 position at Spectra Tech, Inc.?
Responsibilities:
* Advise on the Databricks reference architecture covering Lakeflow, Delta Lake, Unity Catalog Mosaic AI, Databricks SQL, and Databricks Apps
* Provide hands-on guidance on Databricks full-stack solution design
* Advise on design of both data movement (e.g. batch, real-time streaming) and zero-copy patterns
* Guide platform migrations from legacy SQL and document stores onto Databricks and related cloud data platforms
* Recommend security, governance, and compliance-as-code: Unity Catalog ABAC, governed tags, row filters/column masks, key management, networking security and compliance security profiles.
* Advise on design of segregated/tiered data pool architectures aligned to the organization's data classification policy and applicable regulatory requirements, including cross-tier data movement
* Recommend FinOps and performance standards, workload and cost management
* Mentor data engineers, analysts, and AI/ML teams participate in architecture reviews and contribute to standards
* Work with information & cyber security, and other teams as needed for compliance to policies and standards
* Evaluate emerging Databricks capabilities and recommend adoption strategies, including endpoint integrations with third parties as they evolve
Required:
* MS, 20 years designing, building, and operating Databricks workloads with medallion layer architecture
* Hands-on experience with Delta Lake, Unity Catalog,, Spark Structured Streaming, Lakeflow (or DLT/Live Tables), Databricks SQL, MLflow, and Mosaic AI Model Serving
* Strong experience in cloud architecture supporting the platform (VPC/VNet, Security, S3/ADLS, KMS, etc.)
* Proven experience migrating enterprise data workloads from legacy/on prem data warehouse/lake to Databricks
* Production streaming and CDC experience
* Depth in security and governance: Unity Catalog policies (ABAC), classification, encryption, access auditing, compliance-to-control translation
* Hands-on experience designing multi-tier Databricks architectures driven by data classification requirements
* Strong Python and SQL, Spark, Notebooks
* Excellent verbal and written communication skills
Nice to Have:
* Active Databricks certifications
* GenAI and agent systems on Databricks experience
* Delta Sharing and Databricks Clean Rooms for controlled, governed data exchange across organizational/cross-tier/system boundaries
* Familiarity with Oracle EBS and SAP enterprise data
* Experience with SAP BDC/DataSphere and Databricks integration to SAP BDC, with zero-copy patterns
* Lakeflow Designer, Databricks Apps, Genie, Databricks One
* CI/CD, Infrastructure-as-Code (e.g. Terraform) for data platforms
* Experience in designing, building, supporting BI reporting layers
* Data Quality and Data Governance experience
* Prior experience in regulated industries (e.g. utilities, life sciences, government services) and exposure to related frameworks (e.g. FERC/NERC, HIPAA, GxP, GDPR, FedRAMP, ITAR, etc.)
Spectra Tech’s policy is to give equal opportunity to all qualified persons without regard to race, color, religion, sex, age, national origin, individuals with a disability, genetic information, sexual orientation, protected veteran status, or any other basis prohibited by applicable law.