What are the responsibilities and job description for the Data Engineer position at Custom Business Solutions, Inc.?
Summary:
We are looking for a Data Engineer to join our Data Engineering Team. The ideal candidate should have a minimum of 3 years of data engineering experience with excellent analytical reasoning and critical thinking skills. The candidate will be a part of a team that does general extraction, transformation, and load (ETL) work, along with traditional Enterprise Data Warehousing (EDW) work. The Data Engineering Team also creates data pipelines that use change data capture (CDC) mechanisms to move data from on-premises to cloud-based destinations.
Responsibilities:
• Participates in the analysis and development of technical specifications, programming, and testing of Data Engineering components.
• Participates in creating data pipelines and ETL workflows to ensure that design and Team programming standards and guidelines are followed. Assists with updating the Team standards when gaps are identified.
• Follows technology best practices and standards and escalates any issues as deemed appropriate. Follows architecture and design best practices (as guided by the Lead Data Engineer, BI Architect, and Architectural team.)
• Responsible for ETL workflows that utilize Informatica CDI-PC (PowerCenter) to populate an on-premises Enterprise Data Warehouse (EDW). As well as ETL workflows that create flat file extracts.
• ETL workflows that utilize PL/SQL based solutions to populate data-marts for enterprise reporting.
• Configuration and scripting to implement fully automated data pipelines (including but not limited to stored procedures, functions, and scripting components)
• ETL workflows that allow data to flow from on-premises data sources to cloud-based data platforms (e.g. Snowflake) and application platforms (e.g. Salesforce), where data may be consumed by external Customers.
• Follows Team standards for change control and configuration management.
• Participates in 24-hour on-call rotation in support of the platform.
Required Skills/Qualifications:
• Database Platforms: Oracle, SQL Server, and Snowflake
• OS Platforms: Windows Server and RedHat Enterprise Linux
• Languages and Tools: PL/SQL, T-SQL, Informatica CDI-PC (PowerCenter), Informatica IICS or IDMC,
• Understanding and working knowledge of loading an EDW
• Drive and desire to automate repeatable processes.
• Excellent interpersonal skills and communication, as well as the willingness to collaborate with teams across the organization.
Desired Skills/Qualifications:
Experience creating and maintaining data solutions within Snowflake.