What are the responsibilities and job description for the Databrick Engineer in Reston VA position at Hexaware Technologies, Inc?
Databrick Engineers
Location: Reston, VA
Job Description
- This role focuses on designing, building, and maintaining Databricks pipelines and infrastructure.
- Designing, building, and deploying data pipelines and workflows using Databricks.
- Developing and optimizing Apache Spark applications for data processing and analytics.
- Strong proficiency in Python programming language.,
- Experience with Apache Spark and big data processing frameworks.
- Experience on AWS cloud platform using S3 / EMR / Lambda / SQS / SNS / PostGreSQL
- Integrating Databricks with various data sources and platforms.
- Implementing data quality checks and monitoring processes.
- Managing and configuring the Databricks environment, including clusters and notebooks.
- Troubleshooting and resolving issues related to Databricks performance and functionality.
- Collaborating with data scientists and analysts to implement machine learning models.
- Staying up to date with the latest Databricks features and best practices.
- Familiarity with data warehousing and ETL processes.
- Understanding of machine learning concepts and algorithms.
- Strong problem-solving and analytical skills.