Data Integration Specialist

Job Title: Lead Data Integration Engineer

Work Location & Reporting Address: Peoria, IL 61630 (Onsite-Hybrid

Must Have Skills:

⦁ Snap Logic

⦁ AWS

⦁ Python

⦁ Snowflake

⦁ DataStage

Nice to Have Skills:

⦁ GitHub CI/CD integration

Detailed Job Description:

⦁ Design Responsibilities

Solution & Technical Design

1. Translate business requirements into scalable, maintainable, and secure technical designs across DataStage, SnapLogic, AWS, Python, and Snowflake ecosystems.

2. Create high-level and low-level design documents covering data flows, mappings, process orchestration, and integration patterns.

3. Develop data models, transformation logic, and interface specifications for batch, real-time, and event-driven pipelines.

4. Ensure designs align with enterprise architecture standards, security guidelines, and best practices.

5. Collaborate with product owners, analysts, and architects to finalize solution design and validate technical feasibility.

⦁ Senior Development Responsibilities

Hands-on Development & Implementation

1. Build and enhance ETL/ELT pipelines in IBM DataStage, SnapLogic, and Python.

2. Develop scalable Snowflake SQL, stored procedures, and performance-optimized ELT logic.

3. Implement AWS services (Lambda, S3, SNS, SQS, IAM, CloudWatch,Cloudformation, EventBridge) as part of cloud-native data pipelines.

4. Code reusable frameworks, utilities, and components to accelerate team productivity.

5. Conduct peer code reviews, mentor team members, and establish coding standards.

Quality & Delivery

1. Perform unit testing and support integration/UAT activities.

2. Optimize workflows for performance, cost-efficiency, and maintainability.

3. Ensure version control, CI/CD integration, and documentation are consistently maintained.

4. GitHub CI/CD integration

⦁ Production Support & Issue Resolution Responsibilities

Incident & Problem Management

1. Serve as the point for any production issues across DataStage, SnapLogic, Snowflake, AWS, and Tidal.

2. Responsible for analyzing and fixing production issues in any system.

3. Analyze root causes, identify systemic issues, and propose permanent fixes.

4. Develop automated monitoring, alerting, using Python/AWS.

Stability & Reliability

1. Work closely with operations teams to ensure SLAs, uptime, and performance thresholds are met.

2. Improve recovery processes, failover mechanisms, and operational resiliency.

3. Troubleshoot orchestration and scheduling issues in Tidal Enterprise Scheduler.

⦁ Architecture Input & Technical Leadership Responsibilities

Architecture Contributions

1. Provide architectural guidance for new integrations, data warehouse initiatives, and migration projects.

2. Recommend best practices for cloud adoption, pipeline modernization, and platform evolution.

3. Contribute to reference architectures, standards, reusable patterns, and decision frameworks.

Strategic Technology Guidance

1. Evaluate emerging tools and technologies in data engineering, integration, cloud, and orchestration.

2. Champion automation, observability, data quality, and performance optimization initiatives.

3. Collaborate with enterprise architects and infrastructure teams to ensure future-proof design choices.

⦁ Cross-functional Collaboration Responsibilities

1. Work with QA, business analysts, and project managers to enable smooth delivery.

2. Assist and collaborate with offshore teams to ensure alignment, effective knowledge transfer, and smooth delivery across time zones.

3. Communicate technical concepts clearly to both technical and non-technical stakeholders.

⦁ Tools & Technology Scope

1. ETL/ELT: IBM DataStage, SnapLogic

2. Cloud: AWS (Compute, Storage, Serverless, Orchestration, IAM, Monitoring)

3. Data Warehouse: Snowflake

4. Programming: Python, SQL

5. Scheduling: Tidal Enterprise Scheduler

6. Other: Git, CI/CD tools, DevOps pipelines, monitoring platforms