Databricks Platform Administrator / Engineer

Amtex Systems Inc is an information technology and talent solutions company offering talent and BI consulting to the companies in US for over 25 years.

Our solutions are designed to fill resource gaps, by providing the right candidates who deliver value to the organization. Our propensity to nurture and build strong relationships with our clients helps us better understand their business demands and gives us the ability to provide services that are on time and rise above the rest.

Job Role: Databricks Platform Administrator / Engineer

Duration - 12 Months
Location: 55 Water Street and Edison, NJ
Work Status is Hybrid between the two locations for 3 days in office.

Interview: 2 rounds. 1 ZOOM video meeting and 1 in-person in the office

Must be local candidates in New Jersey to work in the Edison, NJ office and downtown Manhattan office.

Job Summary: We are seeking a skilled Databricks Platform Administrator / Engineer with proven experience supporting modern data platforms in a Property & Casualty (P&C) insurance setting. The ideal candidate will be responsible for managing, configuring, optimizing, and maintaining the Databricks environment to support data engineering, analytics, and machine learning workloads in a secure and compliant manner.

Education: Bachelor's or Master's degree in Computer Science, Engineering, Information Systems, or related field.

Required Qualifications:

• 5 years of experience in data platform engineering or system administration.

• 2 years of hands-on experience with Databricks administration on Azure

• Solid understanding of Apache Spark, Delta Lake, and Lakehouse architecture.

• Proficiency in scripting and automation using Python, Bash, PowerShell, or Terraform.

• Familiarity with data security, audit, and compliance frameworks relevant to P&C insurance (e.g., GDPR, NYDFS, SOC 2).

• Experience with Azure Data Services, ADLS, Key Vault, and Azure DevOps

• Strong understanding of the P&C insurance data ecosystem (policy, claims, underwriting, actuarial, etc.).

Preferred Qualifications:

• Databricks Certified Associate / Professional (Administrator / Data Engineer) certification.

• Experience integrating Databricks with core systems like Guidewire, Duck Creek, or custom P&C platforms.

• Knowledge of industry standards like ACORD data models and ISO data feeds.

• Exposure to tools like Unity Catalog, Purview, Collibra, Informatica, or similar.

Soft Skills:

• Strong problem-solving and analytical skills.

• Ability to work cross-functionally in an agile team environment.

• Excellent communication and stakeholder management skills.

• Experience working in regulated enterprise environments with strong data governance standards.

Key Responsibilities:

• Platform Administration & Management:

  • Manage and administer Databricks workspaces, clusters, jobs, libraries, and notebooks.

  • Implement access controls using Unity Catalog, SCIM, and workspace-level RBAC.

  • Configure and monitor autoscaling, cluster policies, and compute usage to optimize cost and performance.

  • Support platform upgrades, patching, and operational automation.

• Security, Compliance & Governance:

  • Enforce security policies aligned with P&C insurance industry regulations.

  • Integrate with enterprise identity providers (Azure AD, Okta) and manage SSO.

  • Support data governance using tools like Unity Catalog, Immuta, or Collibra.

• DevOps & Automation:

  • Build CI/CD pipelines for notebooks, jobs, and ML workflows using tools like Azure DevOps, GitHub Actions, or Jenkins.

  • Automate platform provisioning and configurations using Infrastructure-as-Code (Terraform, ARM, etc.).

• Monitoring & Troubleshooting:

  • Set up logging, monitoring, and alerting using tools like Azure Monitor, Datadog, or Splunk.

  • Troubleshoot performance issues, job failures, and cluster errors.

• Collaboration & Enablement:

  • Work closely with Data Engineers, Data Scientists, and Business Analysts to ensure optimal platform usage.

  • Develop platform usage guidelines, onboarding documentation, and best practices.

  • Provide technical support and training to users.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.