What are the responsibilities and job description for the Sr DataBricks Engineer position at Jobs via Dice?
SNI Companies has partnered with a growing company in Jacksonville, Florida in search of a Databricks Engineer.
Senior Data Engineer Job Description
Responsibilities:
Senior Data Engineer Job Description
Responsibilities:
- Designing, building, and deploying data pipelines and workflows using Databricks.
- Developing and optimizing Apache Spark applications for data processing and analytics.
- Integrating Databricks with various data sources and platforms.
- Ensure data quality, integrity, and governance across all platforms
- Optimize database performance and manage large datasets efficiently
- Manage and configure all aspects of the Databricks environment.
- Monitor and troubleshoot data pipelines and workflows within Databricks.
- Collaborate with stakeholders and analysts to implement machine learning models.
- Staying up-to-date with the latest Databricks features and best practices.
- Implement and enforce best practices in data engineering, including version control, testing, and documentation
- Mentor junior data engineers and contribute to team knowledge sharing.
- Bachelor's degree in Computer Science, Software Engineering, or related field
- 5 years of experience in data engineering or related roles
- Strong proficiency in at least one programming language (e.g., Python, Java or Scala).
- Experience with Apache Spark and big data processing frameworks.
- Knowledge of cloud platforms like AWS, Azure, or Google Cloud Platform.
- Familiarity with data warehousing and ETL/ELT processes.
- Understanding of machine learning concepts and algorithms.
- Strong problem-solving and analytical skills.
- Experience with Databricks Delta Lake or MLflow.
- Knowledge of infrastructure as code (IaC) tools like Terraform.
- Familiarity with CI/CD pipelines for data and machine learning workflows.
- Experience with containerization technologies like Docker.