3 Sr. Data Engineer -Python, SQL, Spark, AWS Jobs in Beaverton, OR
SORT BY
M
B
BizTek People, Inc. | APA International Placement Consultants
Beaverton, OR | Full Time
$109k-135k (estimate)
6 Months Ago
B
BizTek People, Inc. | APA International Placement Consultants
Beaverton, OR | Full Time
$100k-126k (estimate)
6 Months Ago
Sr. Data Engineer -Python, SQL, Spark, AWS
$109k-135k (estimate)
Full Time
4 Months Ago
Mavensoft Technologies is Hiring a Sr. Data Engineer -Python, SQL, Spark, AWS Near Beaverton, OR
Role: Sr. Data Engineer -Python, SQL, Spark, AWS
Duration: 4 Months (Contract)
Location: Beaverton, OR (100% Remote)
Key Skills: Spark, very strong Python, SQL, AWS, S3, AWS EMR, Snowflake, AirflowJob Description:
Professional capacity, for all the competencies listed for each of the following three subject areas: General Purpose Python Programming, Python has been your primary coding language (daily use) for at least 3 years and authored distributable Python packages (packages which can be built, installed, and distributed using setup tools, pip, and twine) You have a solid understanding of how pip dependency resolution works. You are proficient in authoring and automating unit and integration tests for python packages using (minimally) unit tests, pytest, and tox. You are meticulous about code quality, including readability, know your PEP8 guidelines inside and out, and are capable of authoring code which will pass validation by commonly used static analysis tools including mypy and flake8 Database Design and SQL You are proficient in authoring readable, well-structured, SQL SELECT statements using ISO/ANSI-standard SQL. You have hands-on professional experience in data warehouse design and modelling, including authoring DDL statements. Version Control and CI/CD You have experience with trunk-based development (feature branching) using git for version control, with fully automated deployments (CI/CD). Desired Technical Competencies General Purpose Python Programming You have a deep understanding of python’s standard library and python internals. You understand python memory management, how CPython implements built-in data structures, and which data structures are best suited for different scenarios. You understand and can compare/contrast CPython’s built-in concurrency models, when to use each, and what obstacles might prevent the use of each mechanism Database Design, SQL, and Object Relational Models You are adept at performance-tuning SQL queries for both OLAP and OLTP databases. You understand and are prepared to discuss how and when/where to utilize more esoteric and/or modern SQL features such as window functions and common table expressions. You understand and are prepared to discuss the performance implications of columnar vs relational databases. You have hands-on experience in managing database schema migrations (ideally using SQL Alchemy’s ORM Alembic). Version Control and CI/CD You have experience with trunk-based development (feature branching) using git for version control, with fully automated deployments (CI/CD). Cloud Infrastructure and Amazon Web Services You have hands-on experience using boto3 to interact with Amazon Web Services’ resource APIs, particularly Amazon S3 (Simple Storage Service). You have hands-on experience authoring unit and integration tests utilizing local stack to emulate AWS resources. You have hands-on experience using HashiCorp Terraform to manage cloud infrastructure. You have hands-on experience developing serverless ASGI applications using AWS lambda and AWS API Gateway Web API Server and Client Development You have experience planning and executing the design and development of web APIs using a modern python ASGI framework (preferably FastAPI). You have authored, validated, and maintained OpenAPI documents describing your web APIs accurately. You have experience developing and testing python web API client libraries based on an OpenAPI document. Distributed Computing and Apache SparkYou have experience using Apache Spark for ingestion and manipulation of data sets which are too large to process efficiently in-memory. You have hands-on experience translating algorithms and procedures designed by topical subject matter experts, having varying levels of engineering experience, into well-designed data pipelines. You have experience configuring and tuning Spark clusters to optimize use of computing resources for varying workloads. You understand, and can discuss: when and why to use distributed computing frameworks, such as Apache Spark, versus alternate concurrency models such as asyncio or multiprocessing. Database Design, SQL, and Object Relational Models You have experience modeling databases using SQLAlchemy’s ORM framework. You have experience managing database versions and schema migrations using SQLAlchemy with Alembic.Email your resume to usjobs@mavensoft.com
To learn more about Mavensoft visit us online at http://www.mavensoft.com/
Duration: 4 Months (Contract)
Location: Beaverton, OR (100% Remote)
Key Skills: Spark, very strong Python, SQL, AWS, S3, AWS EMR, Snowflake, AirflowJob Description:
- Engineer data solutions in support of Sustainability reporting and analytics initiatives
- Engage with product owner, analysts, visualization developers, and business partners to understand capability requirements, and to develop and support data solutions based on product backlog priorities Skills Preference will be shown to candidates who can provide a link to their open-source code portfolio (a link to your profile on github.com, bitbucket.com, gitlab.com, or another public VCS is sufficient). Required Technical Competencies
- Proficiency communicating effectively and efficiently within a hybrid remote/in-person team structure:
- meticulous about managing your calendar to accurately reflect your free/busy hours.
- respect, seek to learn digital communications etiquette—including region-specific, industry-specific, and organization-specific etiquette.
- proactively initiate constructive discussions while curating and targeting your communications with respect for your colleagues’ time and schedules
- adept at discovering and navigating the complex bureaucratic resources of a large organization.
Professional capacity, for all the competencies listed for each of the following three subject areas: General Purpose Python Programming, Python has been your primary coding language (daily use) for at least 3 years and authored distributable Python packages (packages which can be built, installed, and distributed using setup tools, pip, and twine) You have a solid understanding of how pip dependency resolution works. You are proficient in authoring and automating unit and integration tests for python packages using (minimally) unit tests, pytest, and tox. You are meticulous about code quality, including readability, know your PEP8 guidelines inside and out, and are capable of authoring code which will pass validation by commonly used static analysis tools including mypy and flake8 Database Design and SQL You are proficient in authoring readable, well-structured, SQL SELECT statements using ISO/ANSI-standard SQL. You have hands-on professional experience in data warehouse design and modelling, including authoring DDL statements. Version Control and CI/CD You have experience with trunk-based development (feature branching) using git for version control, with fully automated deployments (CI/CD). Desired Technical Competencies General Purpose Python Programming You have a deep understanding of python’s standard library and python internals. You understand python memory management, how CPython implements built-in data structures, and which data structures are best suited for different scenarios. You understand and can compare/contrast CPython’s built-in concurrency models, when to use each, and what obstacles might prevent the use of each mechanism Database Design, SQL, and Object Relational Models You are adept at performance-tuning SQL queries for both OLAP and OLTP databases. You understand and are prepared to discuss how and when/where to utilize more esoteric and/or modern SQL features such as window functions and common table expressions. You understand and are prepared to discuss the performance implications of columnar vs relational databases. You have hands-on experience in managing database schema migrations (ideally using SQL Alchemy’s ORM Alembic). Version Control and CI/CD You have experience with trunk-based development (feature branching) using git for version control, with fully automated deployments (CI/CD). Cloud Infrastructure and Amazon Web Services You have hands-on experience using boto3 to interact with Amazon Web Services’ resource APIs, particularly Amazon S3 (Simple Storage Service). You have hands-on experience authoring unit and integration tests utilizing local stack to emulate AWS resources. You have hands-on experience using HashiCorp Terraform to manage cloud infrastructure. You have hands-on experience developing serverless ASGI applications using AWS lambda and AWS API Gateway Web API Server and Client Development You have experience planning and executing the design and development of web APIs using a modern python ASGI framework (preferably FastAPI). You have authored, validated, and maintained OpenAPI documents describing your web APIs accurately. You have experience developing and testing python web API client libraries based on an OpenAPI document. Distributed Computing and Apache SparkYou have experience using Apache Spark for ingestion and manipulation of data sets which are too large to process efficiently in-memory. You have hands-on experience translating algorithms and procedures designed by topical subject matter experts, having varying levels of engineering experience, into well-designed data pipelines. You have experience configuring and tuning Spark clusters to optimize use of computing resources for varying workloads. You understand, and can discuss: when and why to use distributed computing frameworks, such as Apache Spark, versus alternate concurrency models such as asyncio or multiprocessing. Database Design, SQL, and Object Relational Models You have experience modeling databases using SQLAlchemy’s ORM framework. You have experience managing database versions and schema migrations using SQLAlchemy with Alembic.Email your resume to usjobs@mavensoft.com
To learn more about Mavensoft visit us online at http://www.mavensoft.com/
Job Summary
JOB TYPE
Full Time
SALARY
$109k-135k (estimate)
POST DATE
01/19/2024
EXPIRATION DATE
06/11/2024
WEBSITE
mavensoft.com
HEADQUARTERS
Beaverton, OR
SIZE
25 - 50
Related Companies
Show more
Similar Jobs
Popular Articles
Top 10 Things to Avoid Putting on a Job Application
Read Article
How to Describe Your Education in a Resume
Read Article
How to Write a Memorable Cover Letter in 5 Ways
Read Article
4 Personal Finance Tips to Make During Job Transition
Read Article
4 Best Camera Setups for Interview
Read Article
Keeping a Job Search Journal: 9 Types of Journaling You Can Do
Read Article
