What are the responsibilities and job description for the Databricks Platform Engineer position at Edward Jones?
Behind everything we do is our purpose: We partner for positive impact to improve the lives of our clients and colleagues, and together, better our communities and society. We are an innovative, flexible, and inclusive organization that attracts, develops, and inspires performance excellence and a sense of belonging.
People are at the center of our partnership. Edward Jones associates are seen, heard, respected, and supported. This is what we believe makes us the best place to start or build your career.
View our Purpose, Inclusion and Citizenship Report.
¹Fortune 500, published June 2024, data as of December 2023. Compensation provided for using, not obtaining, the rating.
About the Opportunity
Are you passionate about enabling advanced analytics, data science, and AI? Do you thrive at the intersection of cloud engineering and transformative business initiatives? If so, the Data Platform Engineering team is seeking an experienced Databricks Platform Engineer to help drive our enterprise data modernization strategy. This unique opportunity positions you at the forefront of our mission to accelerate advanced analytics, data science, and machine learning across the enterprise by building, managing, and evolving our Databricks platform on Azure.
As a Databricks Platform Engineer, you will architect, implement, and optimize our Azure Databricks environment, supporting mission-critical analytics projects across multiple business units. You'll directly enable high-value initiatives while enhancing platform reliability, scalability, and governance. You will collaborate with data engineering, data science, analytics, and application teams to empower business innovation, drive technical excellence, and support key transformation initiatives.
What you will do:
- Platform Architecture & Management: Design, implement, and manage scalable, secure Databricks platforms including workspace/cluster provisioning, lifecycle management, and resource optimization using Delta Lake and Photon to ensure performance, reliability, and cost efficiency.
- Infrastructure Automation & Monitoring: Automate provisioning and deployment using Terraform, GitLab CI/CD, and scripting (Python, Bash, PowerShell) while building observability dashboards to monitor platform health, performance, and costs.
- Security & Governance: Implement and enforce security policies, access controls, and compliance requirements using Unity Catalog and Databricks security features to maintain enterprise-grade data protection.
- Collaboration & Troubleshooting: Partner with data scientists, ML engineers, and product teams to deploy data pipelines and analytical services while resolving cluster configuration issues, job failures, and performance bottlenecks.
- Platform Innovation & Support: Evaluate and adopt emerging Databricks capabilities, document architecture and best practices, mentor junior engineers, and participate in on-call rotation for rapid incident response and platform stability.
- You will also participate in the Databricks platform on-call support rotation, providing expert assistance and rapid response for high-priority incidents, escalations, and critical service interruptions. This is a vital component of our commitment to platform stability and end-user success.
What experience you will need:
- Bachelor's degree in Computer Science, Information Systems or another applicable field is preferred
- 5 years of experience in Systems Administration
- Minimum 3 years hands-on experience designing, deploying, managing, and supporting Azure Databricks, Azure Data Lake Storage (ADLS), and related data solutions as a Platform Engineer/Administrator, with deep knowledge of Azure cloud infrastructure, resource management, security, and networking.
- Proficiency in automation and scripting tools including Terraform (IaC), Azure CLI, Databricks CLI, PowerShell, Python, REST API, and SQL for infrastructure provisioning, deployment, and platform management.
- CI/CD and version control expertise with Git, GitHub Actions, and related tools to automate data and infrastructure workflows, ensuring reliable and repeatable deployment processes.
- Preferred experience in performance tuning and optimization of Azure ADLS and Databricks clusters, cost optimization strategies, and familiarity with data ingestion tools (Qlik Replicate, Apache Kafka, Matillion ELT, Snowflake).
- Agile development experience working collaboratively in cross-functional product teams with exposure to iterative development practices and modern collaboration frameworks.
- Excellent communication skills with the ability to articulate complex system designs, architectural patterns, and technical solutions clearly to stakeholders across all levels of leadership, from technical teams to executives.
Current INTERNAL home-based associates: While this role is posted as hybrid, if selected and accepted, you may retain your home-based status. Edward Jones intends in good faith to continue offering the role as home-based, though future business or regulatory needs may require on-site work.
Check out our U.S. awards and accolades: Insights & Information Blog Postings about Edward Jones
Check out our Canadian awards and accolades: Insights & Information Blog Postings about Edward Jones
Edward Jones does not discriminate on the basis of race, color, gender, religion, national origin, age, disability, sexual orientation, pregnancy, veteran status, genetic information or any other basis prohibited by applicable law.