Senior DataBricks Developer

Location: Remote

Duration: 12 Months

Job Summary
Preferably candidates within EST

Job Description

We are looking for a Senior DataBricks Engineer to develop robust code, understand and implement data pipelines and perform data transformations within the DataBricks platform. This role requires a combination of technical expertise, strong analytical skills, leadership skills, and a strong understanding of data engineering principles. This is a full-time, remote, contract opportunity.

What You'll Do:

• Work closely with Software Designers to ensure best practices are being followed and offer suggestions for improvements to code proficiency and maintainability
• Occasionally interface with customers to assist with analyzing user needs to determine technical requirements
• Design, build, and maintain scalable and reliable data pipelines using DataBricks
• Develop high-quality code with an emphasis on performance, scalability, and security
• Collaborate with cross-functional teams to understand data requirements and deliver solutions that meet business needs
• Implement data transformations and complex algorithms within the DataBricks environment
• Optimize data processing and refine data architecture to improve system efficiency and data quality
• Mentor junior engineers and contribute to the development of best practices within the team
• Stay up-to-date with emerging trends and technologies in data engineering and cloud computing

• Required:

• 7 years of experience in data engineering or a related field
• Strong hands-on experience with DataBricks, including development of code, pipelines, and data transformations
• Proficient in at least one programming language (e.g., Python, Scala, Java).
• In-depth knowledge of Apache Spark and its integration within DataBricks
• Experience with cloud services (AWS, Azure, or Google Cloud Platform) and their data-related products
• Familiarity with CI/CD practices, version control (Git), and automated testing.
• Excellent problem-solving skills and the ability to work independently or as part of a team
• Bachelor's degree in Computer Science, Engineering, Mathematics, or a related technical field

Physical Demands

• Ability to safely and successfully perform the essential job functions consistent with the ADA and other federal, state and local standards
• Sedentary work that involves sitting or remaining stationary most of the time with occasional need to move around the office to attend meetings, etc.
• Ability to conduct repetitive tasks on a computer, utilizing a mouse, keyboard, and monitor