What are the responsibilities and job description for the Quality Assurance Lead position at Ztek Consulting?
Job Role: Quality Assurance Lead (Knowledge of Python & Pyspark & Hadoop Pipeline(HIVE, Impala scope) mandatory)
Location: Dallas, TX
Job Description:
Must Have Technical/Functional Skills:
- We are seeking a Quality Assurance Lead (Contractor) to drive end-to-end testing strategy and quality engineering practices for large-scale data platforms and distributed systems.
- This role will focus on ensuring data integrity, regulatory compliance, and system reliability, particularly within high-volume batch and streaming environments.
- The ideal candidate brings strong expertise in big data testing, automation frameworks, and data validation, along with experience in workflow orchestration and DevOps-driven QA processes.
Roles & Responsibilities:
- Lead and define the QA strategy, test planning, and execution across complex data and application platforms.
- Design and implement automated testing frameworks using Python and PySpark for large-scale data validation.
- Validate data pipelines across Hadoop ecosystem (HDFS, Hive, Impala, Sqoop) ensuring completeness, accuracy, and consistency.
- Develop end-to-end test scenarios covering batch processing, data ingestion, transformation, and downstream consumption.
- Perform database validation and reconciliation using Oracle PL/SQL and SQL-based techniques.
- Validate machine learning pipelines and models, including experiment tracking via MLflow.
- Monitor and validate workflows orchestrated through Control-M / CA-7 schedulers.
- Collaborate with development teams to embed quality gates into CI/CD pipelines using Jenkins and Git.
- Drive test automation within DevOps pipelines, ensuring seamless integration with build and release processes.
- Perform Unix-based scripting for test automation, data checks, and batch validation.
- Lead defect triage, root cause analysis, and resolution tracking.
- Ensure adherence to regulatory and audit requirements, especially in data-sensitive environments.
- Mentor QA engineers and promote best practices in test automation and quality engineering
Required Skills & Experience:
- 8 years of experience in Quality Assurance / Test Engineering, with leadership responsibilities.
- Strong proficiency in Python and PySpark for automation and data validation.
- Solid experience with big data platforms (Hadoop, Hive, Impala, Sqoop).
- Hands-on expertise in Oracle PL/SQL and relational database validation.
- Experience with workflow schedulers such as Control-M or CA-7.
- Strong Unix/Linux scripting experience.
- Experience integrating QA into CI/CD pipelines using Jenkins and Git.
- Familiarity with DevOps practices and automated pipeline testing
- Experience with container platforms (Docker / OpenShift)
- Strong analytical and debugging skills, especially in data-related issues
Nice-to-Have Skills
- Experience with Actimize or OFSAA FCCM platforms (financial crime / AML domain).
- Exposure to cloud platforms (AWS Glue, Athena, S3, SageMaker)
- Experience testing machine learning models and data science workflows
- Knowledge of data governance, lineage, and regulatory reporting framework
Salary : $100,000 - $110,000