Data Engineer with Python and Scala

Job Title: Data Engineer with Python and Scala

Location: Anywhere in the USA (Remote)
Employment Type: Contract

Job Overview:

We are seeking an experienced Data Engineer to join our growing data team. The ideal candidate will have strong hands-on experience with Python, Scala, and database technologies, and will be responsible for building and optimizing scalable data pipelines and data models that power analytics and business insights.

Responsibilities:

• Design, build, and maintain scalable data pipelines and ETL processes.
• Develop and optimize data ingestion, transformation, and loading workflows from multiple sources.
• Collaborate with data analysts, data scientists, and other engineers to support data requirements.
• Ensure data quality, reliability, and performance across all systems.
• Work with large-scale datasets and ensure efficient data processing and storage.
• Implement best practices for data engineering, including code versioning, testing, and CI/CD.
• Contribute to the design and development of data models and metadata management.
• Troubleshoot and optimize queries and data processing jobs.

Required Skills and Qualifications:

• Bachelor's or Master's degree in Computer Science, Engineering, Information Systems, or related field.
• 8 years of experience as a Data Engineer or similar role.
• Strong programming skills in Python and Scala.
• Experience with relational and NoSQL databases (e.g., PostgreSQL, MySQL, MongoDB, Cassandra).
• Strong understanding of ETL concepts, data modeling, and data architecture.
• Hands-on experience with big data technologies such as Spark, Hadoop, or Kafka is a plus.
• Proficiency in SQL and query optimization.
• Experience with cloud platforms (AWS, Azure, or Google Cloud Platform) for data storage and pipeline orchestration.
• Strong problem-solving, debugging, and communication skills.

Preferred Skills:

• Experience with data orchestration tools (Airflow, Luigi, Prefect, etc.).
• Knowledge of streaming data and real-time processing.
• Familiarity with CI/CD pipelines and containerization (Docker, Kubernetes).
• Exposure to data governance and security best practices.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.