SUMMARY

The Sr Data Engineer will need to be analytical, technically inclined, detail-oriented, visually focused, and a self-learner. As a Subject Matter Expert in technical requirements, this role is critical in supporting and implementing data projects and solutions, and effectively engaging with users and other IT staff. As a member of the Business Intelligence team, the individual will be able analyze the existing data warehouse database to ensure the efficiency and accuracy to business functions and operational reporting, as well as modify the ETL processes for a complex and maturing data warehouse. They should also:

• Have an understanding of data modeling, have excellent analytical skills, and the ability to identify solutions to complex data problems.
• Have an understanding or working experience with BI tools such as Cognos or Tableau
• Be able to communicate appropriately with the team leadership.
• Be able to interpret business requests and create technical requirements, including mapping documents to support the data ingestion and pipeline development process, and the execution of ETL jobs.

ESSENTIAL DUTIES AND RESPONSIBILITIES

• Interpret business requests and create technical requirements, including mapping documents to support the data ingestion and pipeline development process, and the execution of ETL jobs.
• Supporting, optimizing and monitoring current data warehousing, data modeling, data integration, business intelligence and ETL processes on NETEZZA platform with CTAS, CBT, zone map and work-load management.
• Provide assistance in troubleshooting and resolving data and system issues, stepping in when necessary to address outages and challenges.
• Develops ETL’s using mostly DataStage with some flavors of SSIS to facilitate Extraction, Transformation and Data Load from disparate systems.
• Maintain methods and procedures for tracking data quality, completeness, redundancy, and improvement.
• Develop and maintain SQL stored procedures and views as part of a star database schema on NETEZZA using WinSQL, NZLOAD, NZSQL and NZBACKUP utilities.
• Need to have the ability to analyze and trouble issues across various applications from Access Tools(Cognos, Tableau and Excel), Data warehouse/Data Marts on Netezza Platform.
• Knowledge of wrapper scripts for Netezza Loader (i.e. NZLOAD) and reporting using Shell scripting, PERL and NZSQL.
• Have an understanding of scheduling tools such as: Active Batch, Tidal or similar
• Troubleshoot issues such as connections, failed jobs, application errors, server alerts, space thresholds, etc. within predefined SLAs.
• Actively participate in design discussions, code reviews, and project-related team meetings.
• Develop processes that facilitate data transformation, manage data structures, metadata, dependencies, and workload management.
• Collaborate with business users to understand functional and data requirements, contributing to the enhancement of data models and pipelines.
• Apply expert-level analytical and problem-solving skills to troubleshoot and resolve technical issues.
• Develop, maintain, and continuously enhance scalable data pipelines, while also developing new data source integrations to support the growing volume and complexity of data.
• Proactively maintain and tune SQL code.
• Identify and capture lessons learned, best practices, and information gained during performance of tasks.
• Be part of a 24x7 on-call rotation for after hour and weekend production support.

EDUCATION and/or EXPERIENCE

Education

• Bachelor’s degree (B. A.) from four-year college or university; or one to two years related experience and/training; or equivalent combination of education and experience.

Experience

• 5-7 years’ experience extracting and summarizing data from a variety of data source types using ETL tools
• Experience in writing complex SQL using Netezza and SQL Server with a strong competency in T-SQL
• Experience interfacing with business users and understanding their requirements
• Experience with DataStage, QualityStage, DataDirector, SSIS, SSRS, Powershell, Active Batch
• Experience with Tableau and Cognos a plus