What are the responsibilities and job description for the Data Engineer position at DMSi Software?
As part of our evolving Product strategy, DMSi seeks to expand our footprint of usable data at scale. DMSi creates and operates technology solutions to support the Building Materials industry. We want to offer uses for our data which serve future product initiatives, customer and business self-service needs, and broader industry needs. All the while, we intend to produce a cloud-based, multi-tenant, data environment which is responsive to the evolving nature of the data landscape and the industry.
The Data Engineer role is laying the foundation for a greenfield platform which will set the tone for the way we approach Business Intelligence, Machine Learning and predictive analysis, and self-service data analytics.
This position includes aspects of architecture, data governance, and data analytics, but firstly encompasses software development and DevOps.
RESPONSIBILITIES AND DUTIES:
- Design and build solutions to support our growing product directions initiatives, solutions which might include:
- Analytics
- Data Visualization
- Reporting
- Self-Service
- Identify, implement, create, sustain, and evolve the DMSi Data environment in support of DMSi’s evolving product, including activities for:
- Data extraction and preparation
- Building and maintaining data lakes
- Choosing and implementing reporting databases
- Scalable data delivery
- Multi-tenancy
- Assure aspects of data quality:
- Availability
- Usability
- Consistency
- Integrity
- Security
- Implement monitoring for pipeline performance and other pipeline metrics.
- Embody both documentation-driven and test-driven development and apply design principles.
- Within a cross-functional agile team, collaborate with related roles and in alignment with Product, Design, Architecture, and Tech Engineering (operations).
- Work closely within the team to prepare data for predictive and prescriptive modeling.
- Evaluate tools and frameworks and articulate the considerations in buy vs. build options.
- Write high quality, testable code. Develop a testing strategy and build appropriate pipelines to enable rapid feedback.
KNOWLEDGE, SKILLS, AND ABILITIES:
Hands on experience in production data pipeline development, automation, and implementation over the last three or more years with:
- Programming Languages commonly used in data engineering such as;
- Python (preferably in AWS)
- TypeScript
- SQL
- Scripting (Unix/Windows)
Experience curating data from multiple sources and devising strategy for use of legacy data stores.
DevOps, Continuous Integration and Continuous Delivery experience.
Knowledgeable with cloud tech stacks (e.g. AWS: S3, Lake Formation, Glue, Athena. Redshift).
Knowledge of building, scaling, and securing multi-tenant environment.
Experience creating recommendations for tools, frameworks, and component builds.
Develops unit tests and uses version control (git).
Experienced in participating in peer code reviews.
Adheres to code quality standards.
Ability to work with large data sets.
Experience with a handful of modern data pipeline tools and the ability to recommend the right tool for the job (Meltano, Singer, dbt, Airflow).
EDUCATION AND EXPERIENCE:
Bachelor or Master’s Degree in Computer Science, Computer Engineering, Electrical Engineering, Management Information Systems, or related field preferred.
Well-experienced in working in Agile shop.
Experience as a software engineer with the bulk – or at least most recent – of those years spent in a modern development environment with CI/CD, source control management, and containerization.
WORK ENVIRONMENT AND PHYSICAL DEMANDS:
Normal office environment with use of computers and telephone systems; no unusual physical demands.
Travel to customer locations including overnight, business air travel, and car rental.
