Senior Data Engineer

Overview

Summary

We are seeking an exceptional and technically strong Sr. Data Engineer with a proven ML experience to be a part of an elite data team driving the next generation of data and AI capabilities using cutting-edge open-source technologies and Databricks ecosystem tools. The right candidate will have substantial hands-on skills from ETL development to feature engineering and MLOps enablement in collaboration with the infrastructure and cloud engineering teams. The candidate should have demonstrated ability to quickly ramp up into new teams and ecosystems with a proactive approach.

Responsibilities

What You'll Do:

• Develop, test, and maintain data pipelines, data APIs and other data services to support a data mesh architecture for application modernization into microservices frameworks.
• Collaborate with data analysts and business stakeholders to understand reporting requirements and deliver effective solutions.
• Develop and operationalize data testing automation using tools like Great Expectations in collaboration with the test automation teams.
• Automate data workflows to improve efficiency and accuracy.
• Automate the monitoring and alerting for data pipeline performance and troubleshoot issues promptly.
• Ensure compliance with data governance and security standards.
• Drive adoption of emerging data engineering, reporting and AI/ML technologies to deliver superior and quantifiable performance gains and new capabilities induction.

Qualifications and Education

Required Qualifications:

• Bachelor’s or Master’s degree in computer science, Information Technology, or related field and 8-10 years of experience
• Proven strong hands-n experience as a Sr, Data Engineer with a strong focus on data-pipeline and reporting solutions working independently
• Proficiency in incorporating Python data tools, utilities and libraries for a superior product outcome
• Experience with AI/ML model deployment and integration.
• Experience of working in an AWS containerized environment with limited container level access.
• Superior skills and a high level of proficiency in Python scripting and SQL scripting
• Proven experience with ETL/ELT tools and frameworks (e.g., Apache Airflow, Prefect, etc.)
• Hands-on experience with data warehousing solutions (e.g., Redshift, AWS DataLake, Delta Lake).
• Experience with reporting and visualization tools (e.g., Tableau, Quicksight).

Compensation

The proposed salary range for this role is $****** to $******* USD. The salary range provided is a good faith estimate representative of all experience levels. Karsun considers several factors when extending an offer, including but not limited to, the role, function and associated responsibilities, a candidate’s work experience, location, education/training, and key skills.