161 sr data engineer Jobs in minneapolis, mn
Contech Systems is Hiring a Senior Data Engineer Near Minneapolis, MN
Title: Sr. Data Engineer
Duration: 6 Months Contract
Location: Remote
Job Description:
You will be part of the team who is focused on building a cutting-edge data analytics platform to support reporting requirements for the business. As a Senior Data Engineer, you will be responsible for the development of complex data sources and pipelines into our data platform (i.e. Snowflake) along with other data applications (i.e. Azure, Airflow, etc.) and automation.
This is a fully remote role based in the United States. Your counterpart team is located in Dublin, Ireland office. While there is no requirement to work in shift hours, there might be an occasional call with Dublin team which can require flexible working.
Primary Responsibilities:
Create & maintain data pipelines using Azure & Snowflake as primary tools
Create SQL Stored procs, Macros to perform complex transformation
Creating logical & physical data models to ensure data integrity is maintained
CI CD pipeline creation & automation using GIT & GIT Actions
Tuning and optimizing data processes
Design and build best in class processes to clean and standardize data
Code Deployments to production environment, troubleshoot production data issues
Modelling of big volume datasets to maximize performance for our BI & Data Science Team
Create Docker images for various applications and deploy them on Kubernetes
Required Qualifications:
Computer Science bachelor's degree or similar
Min 3-6 years of industry experience as a Hands-on Data engineer
Excellent communication skills
Excellent knowledge of SQL, Python
Excellent knowledge of Azure Services such as Blobs, Functions, Azure Data Factory, Service Principal, Containers, Key Vault etc.
Excellent knowledge of Snowflake - Architecture, best practices
Excellent knowledge of Data warehousing & BI Solutions
Excellent Knowledge of change data capture (CDC), ETL, ELT, SCD etc.
Knowledge of CI CD Pipelines using GIT & GIT Actions
Knowledge of different data modelling techniques such as Star Schema, Dimensional models, Data vault
Hands on experience on the following technologies:
o Writing complex SQL querieso Building ETL/ELT/data pipelines using SCD logico Exposure to Kubernetes and Linux containers (i.e. Docker)o Related/complementary open-source software platforms and languages (e.g. Scala, Python, Java, Linux)
Previous experience with Relational Databases (RDBMS) & Non- Relational Database
Analytical and problem-solving experience applied to a Big Data datasets
Good understanding of Access control and Data masking
Experience working in projects with agile/scrum methodologies and high performing team(s)
Exposure to DevOps methodology
Data warehousing principles, architecture and its implementation in large environments
Very good understanding of integration with Tableau
Preferred Qualifications:
Design and build data pipelines (in Spark) to process terabytes of data
Very good understanding of Snowflake integration with data visualization tool such as Tableau
Orchestrate in Airflow the data tasks to run on Kubernetes/Hadoop for the ingestion, processing and cleaning of data
Terraform knowledge and automation
Create real-time analytics pipelines using Kafka / Spark Streaming
Work on Proof of Concepts for Big Data and Data Science
Understanding of United States Healthcare data.
Job Summary
Full Time
$101k-129k (estimate)
05/01/2024
05/20/2024
contech-it.com
TOMS RIVER, NJ
25 - 50
1998
JOSEPH CONTINO
$5M - $10M
Business Services
Related Companies
About Contech Systems
Since 1998, Contech Systems delivered staffing solutions for Fortune 500 companies. Customers value us for our ability make hiring IT talent easy and effective. We offer full service - Recruitment as a Service (RaaS) - Staff augmentation with RTH - Pass through services - Apprentice training to accelerate learning and development Value to you - Save time by leveraging our rigorous vetting process to yield highly qualified candidates - You waste less time by eliminating fraudulent candidates - Leverage a flexible right to hire workforce - Full-service supplier (RaaS)
The following is the career advancement route for Senior Data Engineer positions, which can be used as a reference in future career path planning. As a Senior Data Engineer, it can be promoted into senior positions as a CAE Designer IV that are expected to handle more key tasks, people in this role will get a higher salary paid than an ordinary Senior Data Engineer. You can explore the career advancement for a Senior Data Engineer below and select your interested title to get hiring information.
If you are interested in becoming a Senior Data Engineer, you need to understand the job requirements and the detailed related responsibilities. Of course, a good educational background and an applicable major will also help in job hunting. Below are some tips on how to become a Senior Data Engineer for your reference.
Step 1: Understand the job description and responsibilities of an Accountant.
Quotes from people on Senior Data Engineer job description and responsibilities
The Senior Data Engineer builds data processing frameworks that handle the business’s growing database.
03/09/2022: Billings, MT
The Senior Data Engineer owns and extends the business’s data pipeline through the collection, storage, processing, and transformation of large data-sets.
01/05/2022: Melbourne, FL
Create architecture for real-time, big data applications with minimal maintenance effort.
03/23/2022: Benton Harbor, MI
Work with both structured and unstructured data, with focus on ETL (Extract, Transfer, Load) and stabilizing pipelines and code.
01/22/2022: Greenville, MS
Provides successful deployment and provisioning of data solutions to production or other required environments.
01/12/2022: Dothan, AL
Step 2: Knowing the best tips for becoming an Accountant can help you explore the needs of the position and prepare for the job-related knowledge well ahead of time.
Career tips from people on Senior Data Engineer jobs
Apply for an entry-level data engineer role and gain experience.
02/06/2022: Providence, RI
Earn a bachelor’s degree related to data engineering and begin working on projects.
02/16/2022: Denver, CO
Consider pursuing additional professional engineering or big data certifications.
03/06/2022: Raleigh, NC
Gain knowledge with new data architecture principles and concepts, thorough understanding of layered systems architectures.
01/24/2022: Abilene, TX
Senior data engineers should also have an understanding of other programming languages that help with statistical analysis and modeling, such as Python.
01/02/2022: New York, NY
Step 3: View the best colleges and universities for Senior Data Engineer.
