Lead Data Quality Engineer

Dice is the leading career destination for tech experts at every stage of their careers. Our client, Javen Technologies, Inc, is seeking the following. Apply via Dice today!

If you are interested in below direct client position, please send your resume.

Job Title: Sr. Data Quality Engineer (Databricks/ Spark/ Delta Lake)

Location: Chicago, IL (onsite)

Job Description

Testing in Databricks

Pytest - Framework (Currently using)

Automated test

Test Data from Source system to landing zone

currently No one internally

API Testing

Data testing Pytest Data Transforming (Then fine with no Databricks)

Python or Pytest********

Databricks Python Sql is fine without Pytest

Required Qualifications:

• 5 years of experience in data engineering, data quality engineering, or related roles
• Strong hands-on experience with Databricks, Spark (PySpark), and Delta Lake
• Proven experience implementing data quality frameworks and controls in modern data platforms
• Advanced SQL and data profiling/validation skills
• Experience working with large-scale datasets in cloud environments (AWS or Azure)
• Experience integrating data quality into ELT/ETL pipelines and orchestration tools
• Strong understanding of data governance and data lifecycle management
  
  

Preferred Qualifications:

• Experience in financial services or regulated environments
• Familiarity with data governance tools (e.g., Collibra)
• Experience with data observability or quality tooling (e.g., Monte Carlo, Great Expectations, Deequ, or similar)
• Experience with real-time data quality validation (streaming pipelines)
• Knowledge of regulatory reporting and data controls frameworks
• Cloud or Databricks certifications
  
  

Technical Skills:

• Databricks (Lakehouse, Unity Catalog, workflows)
• Spark / PySpark
• SQL (advanced)
• Delta Lake
• Data quality frameworks (rule engines, validation patterns)
• Data observability and monitoring
• Cloud platforms (AWS or Azure)
• Orchestration tools (Airflow, Control-M)
• APIs and data integration
• CI/CD and DevOps
• Data modeling and lineage concepts