What are the responsibilities and job description for the Big Data Engineer position at K&K Global Talent Solutions?
K&K Global Talent Solutions Inc. is an international recruiting agency that has been providing technical resources in the Canada and the USA region since 1993.
This position is with one of our clients in USA, who is actively hiring candidates to expand their teams.
Job Description
Must Have Technical/Functional Skills
• Strong experience working with Quantexa platform (development and configuration).
• Hands-on expertise in Scala programming.
• Experience with Quantexa Compound Key configuration and entity resolution frameworks.
• Proficiency in PySpark and big data processing.
• Experience in data pipeline setup and data engineering workflows.
• Good understanding of distributed systems and large-scale data processing.
• Strong problem-solving and analytical skills.
Roles & Responsibilities
- Lead and contribute to end-to-end design and development of Quantexa-based solutions.
- Perform hands-on development using Scala and PySpark within the Quantexa ecosystem.
- Configure and implement Quantexa Compound Keys and data models for entity resolution.
- Design and build robust data pipelines, including ingestion, transformation, and processing of large datasets.
- Collaborate with stakeholders to understand business requirements and translate them into technical solutions.
- Optimize system performance and ensure scalability of Quantexa applications.
- Provide guidance, best practices, and technical mentorship to development teams.
- Ensure adherence to data governance, security, and compliance standards.