What are the responsibilities and job description for the Lead Business Analyst - Databricks position at Databricks - Energy Transfer Family of Partnerships Careers?
Energy Transfer, recognized by Forbes as one of America's best large employers, is dedicated to responsibly and safely delivering America’s energy. We are driven to inspire our employees to create superior value for our customers, our investors, a sustainable future and giving back to the community where we have long-standing commitments to causes including MD Anderson Children’s Cancer Hospital, The Salvation Army, The American Red Cross, Ronald McDonald House and many more.
We value all of our employees who make our growth and success possible. We are proud to offer industry leading compensation, comprehensive benefits, 401(k) match with additional profit sharing, PTO and abundant career opportunities.
Come join our award winning 12,000 strong organization as we fuel the world and each other!
Summary
We are looking for a highly technical Databricks Lead with strong hands-on experience in designing and delivering enterprise data solutions
Key Responsibilities:
- Solution architecture and design documents.
- Medallion architecture implementation across data pipelines.
- Working Databricks notebooks, jobs, workflows, and reusable code assets.
- Data pipeline standards and reference patterns.
- Performance, scalability, and cost optimization recommendations.
- Technical design review inputs and architecture decisions.
- Issue resolution plans for production and performance problems.
- Clear stakeholder communication on design and delivery progress.
Success in the Role
- Databricks solutions are delivered on time and meet business needs.
- Pipelines are reliable, scalable, and supportable.
- Data quality and operational controls are in place.
- Production issues are resolved quickly with minimal disruption.
- Architecture decisions are documented and reusable.
- Team capability improves through leadership and mentoring.
Education and/or Experience, Knowledge, Skills & Abilities:
To perform this job successfully, an individual must be able to perform each essential job duty satisfactorily. The requirements for this position are listed below:
Required Qualifications:
- Bachelor's degree or equivalent and 8 years of relevant work experience
- Strong hands-on experience with Databricks in enterprise environments.
- Experience leading multiple end-to-end Databricks implementations.
- Proven expertise in medallion architecture, including Bronze, Silver, and Gold layers.
- Strong working knowledge of Spark, Delta Lake, SQL, Python, and data pipeline design.
- Experience with data ingestion, transformation, orchestration, and data quality controls.
- Ability to troubleshoot complex production issues and optimize performance.
- Experience translating business and functional needs into technical solutions.
- Strong understanding of architecture standards, reusable frameworks, and best practices.
- Experience collaborating with business stakeholders, architects, and engineering teams.
- Exposure to deployment planning, code reviews, release management, and production support.
- Ability to lead technical discussions and mentor team members.
Preferred Qualifications:
- Experience with Azure, AWS, or GCP.
- Familiarity with Unity Catalog, Delta Sharing, and governance practices.
- Experience with CI/CD, automated testing, and release pipelines.
- Background in data modernization or migration programs.
- Exposure to analytics, data warehousing, or machine learning use cases
Working Conditions:
The work environment characteristics described here are representative of those an employee encounters while performing the essential functions of this job.
- Usually, normal office working conditions.
- Must be able to remain in a stationary position 50% of the time due to prolonged periods of sitting or standing.
- Occasional overnight travel may be required.