What are the responsibilities and job description for the Data Engineer position at Thinking Minds Inc.?
Job Summary
Location: United States
Experience: 5 years in Data Engineering
Job Type: Permanent
Thinking Minds' client is looking for an experienced Data Engineer to expand and optimize their data and data pipeline architecture. This role is crucial for optimizing data flow and collection across cross-functional teams. The successful candidate will support software developers, data architects, and data analysts in data initiatives, enhancing the company's data warehouse to support next-generation products.
Ideal Candidate Should Have
Experience with ETL processes
Proficiency in SQL and complex query writing
Strong understanding of data warehousing
Ability to work in mixed cloud and on-prem environments
Skills in data integration and transformation
Role Overview
The Data Engineer will be responsible for building both real-time and batch data integrations from various source systems into the enterprise data warehouse. They will work closely with data and analytics experts to enhance the functionality of data systems. The role involves assembling large, complex data sets to meet business requirements, designing and implementing real-time ETL and batch processing, and providing production support for data pipelines.
Key Responsibilities
Build and optimize data pipelines and databases
Write complex SQL queries for reporting
Assemble large, complex data sets
Design and implement real-time ETL processes
Provide production support for data integration
Scale and tune data pipelines
Develop algorithms to improve data utility
Collaborate with data architects and analysts
Top Skills and Modules
ETL processes
SQL
Data warehousing
Data integration
Real-time and batch processing
Data pipeline optimization
Cloud and on-prem environments
Location: United States
Experience: 5 years in Data Engineering
Job Type: Permanent
Thinking Minds' client is looking for an experienced Data Engineer to expand and optimize their data and data pipeline architecture. This role is crucial for optimizing data flow and collection across cross-functional teams. The successful candidate will support software developers, data architects, and data analysts in data initiatives, enhancing the company's data warehouse to support next-generation products.
Ideal Candidate Should Have
Experience with ETL processes
Proficiency in SQL and complex query writing
Strong understanding of data warehousing
Ability to work in mixed cloud and on-prem environments
Skills in data integration and transformation
Role Overview
The Data Engineer will be responsible for building both real-time and batch data integrations from various source systems into the enterprise data warehouse. They will work closely with data and analytics experts to enhance the functionality of data systems. The role involves assembling large, complex data sets to meet business requirements, designing and implementing real-time ETL and batch processing, and providing production support for data pipelines.
Key Responsibilities
Build and optimize data pipelines and databases
Write complex SQL queries for reporting
Assemble large, complex data sets
Design and implement real-time ETL processes
Provide production support for data integration
Scale and tune data pipelines
Develop algorithms to improve data utility
Collaborate with data architects and analysts
Top Skills and Modules
ETL processes
SQL
Data warehousing
Data integration
Real-time and batch processing
Data pipeline optimization
Cloud and on-prem environments