Recent Searches

You haven't searched anything yet.

41 Direct Client Sr Lead Data Engineer Apache Spark, Spark Core, Spark SQL Data Bricks Jobs in Seattle, WA

SET JOB ALERT
Details...
The AES Group
Seattle, WA | Full Time
$110k-130k (estimate)
5 Days Ago
Social Finance (SoFi)
Seattle, WA | Full Time
$143k-170k (estimate)
3 Days Ago
Social Finance (SoFi)
Seattle, WA | Full Time
$131k-157k (estimate)
4 Days Ago
ChaTeck
Seattle, WA | Full Time
$125k-154k (estimate)
3 Weeks Ago
ChaTeck
Seattle, WA | Full Time
$125k-154k (estimate)
3 Weeks Ago
Qumulo Careers
Seattle, WA | Full Time
$123k-148k (estimate)
3 Weeks Ago
PitchBook Data, Inc.
Seattle, WA | Full Time
$125k-154k (estimate)
2 Months Ago
Mokshaa llc
Seattle, WA | Full Time
$115k-145k (estimate)
1 Week Ago
Amazon
Seattle, WA | Full Time
$114k-144k (estimate)
2 Weeks Ago
W3Global Inc
Seattle, WA | Full Time
$120k-151k (estimate)
2 Weeks Ago
Opala
Seattle, WA | Full Time
$111k-142k (estimate)
2 Months Ago
System1
Seattle, WA | Full Time
$96k-126k (estimate)
3 Months Ago
Coupang Internal
Seattle, WA | Full Time
$163k-200k (estimate)
5 Months Ago
Amazon
Seattle, WA | Full Time
$116k-146k (estimate)
Just Posted
The Walt Disney Company
Seattle, WA | Full Time
$124k-149k (estimate)
2 Months Ago
Only External Postings
Seattle, WA | Full Time
$125k-154k (estimate)
2 Months Ago
Apple
Apple
Seattle, WA | Full Time
$138k-164k (estimate)
9 Months Ago
HireIO, Inc.
Seattle, WA | Full Time
$105k-132k (estimate)
7 Months Ago
Apple
Apple
Seattle, WA | Full Time
$125k-150k (estimate)
11 Months Ago
Amazon
Seattle, WA | Full Time
$124k-149k (estimate)
4 Days Ago
Apple
Apple
Seattle, WA | Full Time
$128k-158k (estimate)
7 Months Ago
Valere Labs Pvt Ltd
Seattle, WA | Full Time
$126k-155k (estimate)
Just Posted
Amazon
Seattle, WA | Full Time
$127k-159k (estimate)
3 Weeks Ago
Apple
Apple
Seattle, WA | Full Time
$128k-158k (estimate)
4 Months Ago
HireIO, Inc.
Seattle, WA | Full Time
$102k-123k (estimate)
7 Months Ago
Apple
Apple
Seattle, WA | Full Time
$125k-152k (estimate)
1 Week Ago
Outreach
Seattle, WA | Full Time
$134k-160k (estimate)
3 Weeks Ago
reddit
Seattle, WA | Full Time
$123k-148k (estimate)
2 Months Ago
Direct Client Sr Lead Data Engineer Apache Spark, Spark Core, Spark SQL Data Bricks
ChaTeck Seattle, WA
$125k-154k (estimate)
Full Time | IT Outsourcing & Consulting 3 Weeks Ago
Save

ChaTeck is Hiring a Direct Client Sr Lead Data Engineer Apache Spark, Spark Core, Spark SQL Data Bricks Near Seattle, WA

Job Details

Direct Client - Sr Lead Data Engineer Apache Spark, Spark Core, Spark SQL, Spark StreamingData Bricks
Location: Hybrid (Seattle, WA)
Length: 12 Months

Job description
This position contributes to Client success by building enterprise data services for analytic solutions. This position is responsible for design, development, testing and support for data pipelines to enable continuous data processing for data exploration, data preparation and real-time business analytics. Models and acts in accordance with Client guiding principles.

Tops 3 Skills Needed

1

Data Engineering

5 years

2

Data Bricks

5 years

3

Sparks/ Dev ops practices

5 years

Years of Experience:

  • 5 years

Technology requirements:

  • Proficiency in Apache Spark, including Spark Core, Spark SQL and Spark Streaming.
  • Proficiency in languages such as Python for data processing and scripting.

Basic Qualifications/ Experience:

  • Experience in designing and implementing ETL processes using Databricks notebooks for efficient data extraction, transformation, and loading.
  • In-depth knowledge of the Databricks Unified Analytics Platform and its features for collaborative big data analytics.
  • Understanding of data modeling concepts for designing database structures.
  • Proficiency in working with both relational databases and NoSQL databases.
  • Integration of data from diverse sources, including APIs, databases, streaming data, and external data feeds.
  • Implementation of processes to ensure data quality, including data validation, cleansing, and error handling.
  • Knowledge of cluster management, optimization, and scaling for efficient data processing.
  • Optimization of Spark jobs and Databricks clusters for better performance.
  • Proficiency in cloud platforms such as Azure for building scalable and flexible data architectures.
  • Use of tools like Apache Airflow, ADF, or Databricks to orchestrate and schedule data workflows.

Degree or certifications required:

  • Education (minimum education level, degree or certification necessary): Bachelor s degree in computer science, management information systems, or related discipline

Skills (minimum skills required):

  • 5-7 years Architect and design large scale high performance distributed systems
  • 5-7 years SQL Platform
  • 2 years Exposure No-SQL Platform is a plus
  • 5 years Hadoop, YARN, MapReduce, Pig or Hive, Spark
  • 2 years Data platform implementation on Azure or AWS is a plus

Key Responsibilities:

  • Responsibilities and essential job functions include but are not limited to the following:
  • Demonstrate deep knowledge and ability to lead others in the data engineering team to build and support noninteractive (batch, distributed) & real-time, highly available data, data pipeline and technology capabilities
  • Translate strategic requirements into business requirements to ensure solutions meet business needs
  • Work with infrastructure provisioning & configuration tools to develop scripts to automate deployment of physical and virtual environments; to develop tools to monitor usage of virtual resources
  • Assist in the definition of architecture that ensure that solutions are built within a consistent framework
  • Lead resolution activities for complex data issues
  • Define & implement data retention policies and procedures
  • Define & implement data governance policies and procedures
  • Identify improvements in team coding standards and help in implementation of the improvements
  • Leverage subject matter expertise to coordinate issue resolution efforts across peer support groups, technical support teams, and vendors
  • Develop and maintain documentation relating to all assigned systems and projects
  • Perform systems and applications performance characterization and trade-off studies through analysis and simulation
  • Perform root cause analysis to identify permanent resolutions to software or business process issues
  • Lead by example by demonstrating the Client mission and value

Nice-to-Haves:

  • Knowledge of data security best practices and the implementation of measures to ensure data privacy and compliance.
  • Implementation of monitoring and logging solutions to track the health and performance of pipelines.
  • Familiarity with monitoring platforms like DataDog and New Relic
  • Azure

Job Summary

JOB TYPE

Full Time

INDUSTRY

IT Outsourcing & Consulting

SALARY

$125k-154k (estimate)

POST DATE

04/07/2024

EXPIRATION DATE

06/06/2024

WEBSITE

chateck.com

HEADQUARTERS

Bothell, WA

SIZE

25 - 50

INDUSTRY

IT Outsourcing & Consulting

Show more

ChaTeck
Full Time
$96k-118k (estimate)
2 Weeks Ago
ChaTeck
Full Time
$125k-154k (estimate)
3 Weeks Ago