Spark Scala Data Engineer

Role: Spark Scala Data Engineer

Location - Sunnyvale CA or Austin TX

Type: Contract

A Spark Scala Data Engineer is responsible for designing, developing, and optimizing large-scale data processing systems using Apache Spark and Scala. The role involves building ETL pipelines, processing big data, improving data platform performance, and supporting analytics and data science workloads

.Key Responsibilitie

• sDevelop scalable ETL and data processing pipelines using Spark and Scala
• .Work with structured and unstructured datasets
• .Optimize Spark jobs for performance and resource utilization
• .Integrate data from multiple sources such as databases, APIs, and streaming platforms
• .Implement data quality, validation, and monitoring processes
• .Collaborate with data scientists, analysts, and software engineers
• .Deploy and maintain applications in cloud or distributed environments

.Required Skill

• sStrong experience with Scala programming
• .Hands-on experience with Apache Spark (Core, SQL, DataFrames, Structured Streaming)
• .Knowledge of SQL and data warehousing concepts
• .Experience with distributed systems and big data technologies
• .Familiarity with version control tools such as Git
• .Experience with cloud platforms (AWS, Azure, or GCP)
• .Knowledge of workflow orchestration tools such as Airflow

.Preferred Skill

• sExperience with Kafka, Hadoop, Delta Lake, or Databricks
• .Understanding of CI/CD pipelines
• .Knowledge of data modeling and performance tuning