What are the responsibilities and job description for the Data Engineer position at Jobs via Dice?
Description
We are looking for a skilled Data Engineer to join our team in Ann Arbor, Michigan, and contribute to the development of a modern, scalable data platform. In this role, you will focus on building efficient data pipelines, ensuring data quality, and enabling seamless integration across systems to support business analytics and decision-making. This position offers an exciting opportunity to work with cutting-edge technologies and play a key role in the transformation of our data environment.
Responsibilities:
Design and implement robust data pipelines on Azure using tools such as Databricks, Spark, Delta Lake, and Airflow.
Develop workflows to ingest and integrate data from diverse sources into Azure Data Lake.
Build and maintain data transformation layers following the medallion architecture principles.
Apply data quality checks, validation processes, and deduplication techniques to ensure accuracy and reliability.
Create reusable and parameterized notebooks to streamline batch and streaming data processes.
Optimize merge and update logic in Delta Lake by leveraging efficient partitioning strategies.
Collaborate with business and application teams to understand and fulfill data integration requirements.
Enable downstream integrations with APIs, Power BI dashboards, and reporting systems.
Establish monitoring, logging, and data lineage tracking using tools like Unity Catalog and Azure Monitor.
Participate in code reviews, agile development practices, and team design discussions.
Requirements
Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent experience.
Minimum of 5 years of experience in designing and building distributed data systems with attention to detail.
Proficiency in Databricks, Delta Lake, and Azure-based big data tools.
Hands-on experience with PySpark, SQL Server, and T-SQL, including query optimization and performance tuning.
Familiarity with Git, CI/CD processes, and agile development methodologies.
Strong understanding of modern data lake architecture and structured streaming.
Experience in financial services or regulated data environments is preferred.
Excellent problem-solving skills and the ability to collaborate effectively with cross-functional teams.
Technology Doesn't Change the World, People Do.
Robert Half is the world's first and largest specialized talent solutions firm that connects highly qualified job seekers to opportunities at great companies. We offer contract, temporary and permanent placement solutions for finance and accounting, technology, marketing and creative, legal, and administrative and customer support roles.
Robert Half works to put you in the best position to succeed. We provide access to top jobs, competitive compensation and benefits, and free online training. Stay on top of every opportunity - whenever you choose - even on the go. Download the Robert Half app and get 1-tap apply, notifications of AI-matched jobs, and much more.
All applicants applying for U.S. job openings must be legally authorized to work in the United States. Benefits are available to contract/temporary professionals, including medical, vision, dental, and life and disability insurance. Hired contract/temporary professionals are also eligible to enroll in our company 401(k) plan. Visit roberthalf.gobenefits.net for more information.
2025 Robert Half. An Equal Opportunity Employer. M/F/Disability/Veterans. By clicking "Apply Now," you're agreeing to Robert Half's Terms of Use.
We are looking for a skilled Data Engineer to join our team in Ann Arbor, Michigan, and contribute to the development of a modern, scalable data platform. In this role, you will focus on building efficient data pipelines, ensuring data quality, and enabling seamless integration across systems to support business analytics and decision-making. This position offers an exciting opportunity to work with cutting-edge technologies and play a key role in the transformation of our data environment.
Responsibilities:
Design and implement robust data pipelines on Azure using tools such as Databricks, Spark, Delta Lake, and Airflow.
Develop workflows to ingest and integrate data from diverse sources into Azure Data Lake.
Build and maintain data transformation layers following the medallion architecture principles.
Apply data quality checks, validation processes, and deduplication techniques to ensure accuracy and reliability.
Create reusable and parameterized notebooks to streamline batch and streaming data processes.
Optimize merge and update logic in Delta Lake by leveraging efficient partitioning strategies.
Collaborate with business and application teams to understand and fulfill data integration requirements.
Enable downstream integrations with APIs, Power BI dashboards, and reporting systems.
Establish monitoring, logging, and data lineage tracking using tools like Unity Catalog and Azure Monitor.
Participate in code reviews, agile development practices, and team design discussions.
Requirements
Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent experience.
Minimum of 5 years of experience in designing and building distributed data systems with attention to detail.
Proficiency in Databricks, Delta Lake, and Azure-based big data tools.
Hands-on experience with PySpark, SQL Server, and T-SQL, including query optimization and performance tuning.
Familiarity with Git, CI/CD processes, and agile development methodologies.
Strong understanding of modern data lake architecture and structured streaming.
Experience in financial services or regulated data environments is preferred.
Excellent problem-solving skills and the ability to collaborate effectively with cross-functional teams.
Technology Doesn't Change the World, People Do.
Robert Half is the world's first and largest specialized talent solutions firm that connects highly qualified job seekers to opportunities at great companies. We offer contract, temporary and permanent placement solutions for finance and accounting, technology, marketing and creative, legal, and administrative and customer support roles.
Robert Half works to put you in the best position to succeed. We provide access to top jobs, competitive compensation and benefits, and free online training. Stay on top of every opportunity - whenever you choose - even on the go. Download the Robert Half app and get 1-tap apply, notifications of AI-matched jobs, and much more.
All applicants applying for U.S. job openings must be legally authorized to work in the United States. Benefits are available to contract/temporary professionals, including medical, vision, dental, and life and disability insurance. Hired contract/temporary professionals are also eligible to enroll in our company 401(k) plan. Visit roberthalf.gobenefits.net for more information.
2025 Robert Half. An Equal Opportunity Employer. M/F/Disability/Veterans. By clicking "Apply Now," you're agreeing to Robert Half's Terms of Use.