1 DataBricks Developer on AWS Job in Dallas, TX

SORT BY

SET JOB ALERT

OFF

Details...

DataBricks Developer on AWS

Infinite Computer Solutions

Dallas, TX | Full Time

$99k-127k (estimate)

2 Weeks Ago

DataBricks Developer on AWS

$99k-127k (estimate)

Full Time | Wholesale 2 Weeks Ago

Save

Sorry! This job is no longer available. Please explore similar jobs listed on the left.

Infinite Computer Solutions is Hiring a DataBricks Developer on AWS Near Dallas, TX

Job Details

Developing Data Pipelines: Designing and implementing scalable data pipelines using Databricks on AWS infrastructure leveraging Python and Spark.
Data Ingestion: Managing the ingestion of structured and unstructured data from various sources into Databricks clusters on AWS, ensuring data quality and reliability.
Data Processing: Performing data processing tasks, such as data transformation, cleansing, aggregation, and enrichment using Databricks tools and AWS services like EMR (Elastic MapReduce).
Optimization and Performance Tuning: Optimizing Databricks jobs and clusters for performance, efficiency, and cost-effectiveness on the AWS platform.
Integration: Integrating Databricks with other AWS services such as S3, Glue, Redshift, and Lambda to build end-to-end data solutions.
Monitoring and Maintenance: Monitoring Databricks clusters, jobs, and workflows for performance issues, troubleshooting errors, and ensuring high availability and reliability.
Security and Compliance: Implementing security best practices and ensuring compliance with data privacy regulations (e.g., GDPR, CCPA) in Databricks deployments on AWS.
Collaboration: Collaborating with data engineers, data scientists, and other stakeholders to understand data requirements and deliver solutions that meet business needs.
Documentation: Documenting Databricks workflows, architecture, and best practices to facilitate knowledge sharing and maintain system documentation.
Continuous Learning: Keeping up-to-date with the latest advancements in Databricks, AWS services, and data engineering technologies to continuously improve skills and contribute to innovation in data solutions.