Primary Skills: testing, SQL, Azure Databricks, and Spark/PySpark
Description:
We are seeking a skilled Data Quality Engineer to join our team and play a critical role in ensuring the reliability, accuracy, and consistency of data within our warehouse and manufacturing data platform in the Client supply chain space. While the title may suggest a focus on quality assurance, this role requires a blend of data engineering and testing expertise. The Data Quality Engineer will be responsible for implementing robust test frameworks, writing automation test cases, and collaborating closely with data engineers to validate and verify the functionality of data pipelines and processes. This is a key role in maintaining the integrity of our data platform and driving continuous improvement in data quality standards.
Technical Skills:
Proficiency in data engineering technologies such as Azure Databricks, Synapse Analytics, and SQL Hyperscale.
Strong programming skills in Python and PySpark for developing automation test scripts.
Experience with building and implementing test frameworks for data-centric applications in Azure cloud environments.
Knowledge of data quality assurance methodologies and best practices.
Familiarity with version control systems and CI/CD pipelines for automated testing and deployment.
Understanding of data modeling, ETL processes, and data warehousing concepts.
Education: Bachelors Degree
Additional client information: