What are the responsibilities and job description for the Databricks Engineer AWS_Dallas, TX-Locals position at Keylent?
Databricks Engineer AWS
Dallas, TX | Local Only
Work Mode: Onsite - Inperson interview is MUST
Required Skills:
- 10 years of strong hands-on experience with AWS services
- Strong expertise in Python and PySpark development
- Strong hands-on experience with Databricks
- Experience developing Databricks notebooks, workflows, and ETL pipelines
- Strong knowledge of Apache Spark / PySpark and distributed data processing
- Experience with Databricks Delta Lake / Delta tables
- Hands-on experience with Databricks SQL and data transformation
- Experience building and optimizing Databricks ETL/data engineering pipelines
- Experience with AWS Glue and ETL process development
- Strong SQL skills for data transformation, analysis, and optimization
- Hands-on experience with Kafka for real-time data processing
- Experience with Amazon Redshift
- Experience integrating Databricks with AWS services, including S3, Glue, and Redshift
- Knowledge of Git pipelines and CI/CD processes
- Experience deploying and managing Databricks jobs/workflows through CI/CD
- Strong understanding of ETL workflows and data integration
- Experience in ETL testing, validation, and data quality checks
- Expertise in ETL automation frameworks and process optimization
- Experience with data pipeline performance tuning and Spark optimization
- Strong understanding of batch and real-time data processing
- Experience troubleshooting and optimizing Spark/Databricks jobs
- Strong knowledge of data lake / data warehouse concepts
- Experience working with large-volume datasets and distributed processing
Must Have: AWS Python PySpark Databricks AWS Glue SQL Kafka Redshift ETL