Databricks Data Engineer (Remote)

Benefits

Why Join our Team? At Vallen, we embrace what makes us unique. We thrive on the diversity of our associates and the different ways each of us contributes to Vallen's success. We pride ourselves on being an inclusive company that values the growth of our associates. When you join Vallen, you become part of our winning team. Our vast network of connections is here to help you transform your job into a career. Vallen is an industry leader in delivering flexible supply chain solutions while creating value for our customers, our supply partners, and our associates. Your Benefits at Vallen:

• Medical, Dental, Vision
• Paid time off (vacation, holidays, sick days)
• 401K with discretionary company match - (US positions only)
• Employer-paid Basic Life for Employee, Spouse, and Dependents
• Employer-paid Short-Term and Long-Term Disability
• Health Care and Dependent Care Flexible Spending Accounts
• Vitality Wellness Program
• Employee Assistance Program
• Employee Resource Groups for networking and team building
• Tuition Reimbursement Program
• Employee Referral Program
• Safety Shoe and Safety Glasses reimbursement
• Employee Discounts through BenefitHub
• Advancement Opportunity.
  
  

Position Summary

Join the Vallen team as a Senior Data Engineer. We are seeking to fill this role with an individual with 5-7 years of experience in data engineering and a strong background in Databricks. This role is ideal for someone passionate about building scalable data solutions, enabling analytics, and driving innovation in a modern data platform.

Essential Duties And Responsibilities

• Design and develop scalable data pipelines using PySpark, Spark SQL, and Python within the Databricks environment.
• Build modular, version-controlled transformation workflows using DBT (Data Build Tool).
• Package and deploy Databricks workflows and notebooks using Databricks Asset Bundles (DABS) for CI/CD and environment management.
• Integrate Databricks workflows with Azure DevOps for automated testing, deployment, and version control.
• Develop and maintain robust CI/CD pipelines for data engineering workflows using Git, Azure DevOps, and DABS.
• Implement and optimize dimensional data models, ELT/ETL processes, and performance tuning in cloud data platforms.
• Collaborate with data scientists, analysts, and business stakeholders to deliver high-impact data solutions.
• Implement logging, alerting, and monitoring for data pipelines using tools like Databricks Jobs, MLflow, or Azure Monitor.
  
  

Job Qualifications

• 5-7 years of experience as a Data Engineer, with at least 3 years working in Databricks.
• Strong proficiency in Python, PySpark, and Spark SQL.
• Hands-on experience with DBT in a cloud data platform.
• Experience with DABS for workflow packaging and deployment.
• Proven expertise in Azure DevOps, Git, and CI/CD pipeline development.
• Solid understanding of data modeling, ETL/ELT, and performance optimization.
• Experience implementing monitoring and observability for data pipelines.
• Excellent communication and collaboration skills.
  
  

Preferred Qualifications

• Experience in Azure cloud ecosystem.
• Familiarity with MLflow and other observability tools.
• Exposure to data governance and metadata management practices.
  
  

Work Environment & Physical Demands

Reasonable accommodations will be evaluated and may be implemented to enable individuals with disabilities to perform essential functions of this position.

• This job operates in a professional office environment and routinely uses standard office equipment such as computers, phones, photocopiers, filing cabinets, etc.
• The work environment is generally favorable. Lighting and temperature are adequate and there are no hazardous or unpleasant conditions caused by noise, dust, etc.
• Long periods of time working on a computer and performing repetitive key-boarding activities.
• Long periods of time performing work over the phone and participating in group conference calls.
• Occasional evening and weekend work may be required as job duties demand and may include little to no advanced notice.
• Specific vision abilities required by this job include close vision, distance vision, color vision, peripheral vision, depth perception and ability to adjust focus.
• Frequently required to sit and/or stand.
  
  

Stay up to date with Vallen, and follow us on LinkedIn, Facebook and Twitter.

Equal Opportunity Employer - Ethnically and Culturally Diverse/Females/Veterans/Disabled/Sexual Orientation/Gender Identity