What are the responsibilities and job description for the Senior Data & Integration Engineer/Designer/Architect position at Dimension Consulting?
Hi,
Hope you’re doing well.
Please take a moment to review the job description I’ve shared. If you’re interested in this opportunity, kindly send me your updated resume at your earliest convenience
Job Title: Technology Lead | Database Administration | Teradata Administration -- Senior Data & Integration Engineer/Designer/Architect
Work Location & Reporting Address: Peoria, IL 61630
Contract duration: 12
Must Have Skills:
⦁ SnapLogic
⦁ AWS
⦁ Python
⦁ Snowflake
⦁ DataStage
Nice to Have Skills:
⦁ GitHub CI/CD integration
Detailed Job Description:
⦁ Design Responsibilities
Solution & Technical Design
1. Translate business requirements into scalable, maintainable, and secure technical designs across DataStage, SnapLogic, AWS, Python, and Snowflake ecosystems.
2. Create high-level and low-level design documents covering data flows, mappings, process orchestration, and integration patterns.
3. Develop data models, transformation logic, and interface specifications for batch, real-time, and event-driven pipelines.
4. Ensure designs align with enterprise architecture standards, security guidelines, and best practices.
5. Collaborate with product owners, analysts, and architects to finalize solution design and validate technical feasibility.
⦁ Senior Development Responsibilities
Hands-on Development & Implementation
1. Build and enhance ETL/ELT pipelines in IBM DataStage, SnapLogic, and Python.
2. Develop scalable Snowflake SQL, stored procedures, and performance-optimized ELT logic.
3. Implement AWS services (Lambda, S3, SNS, SQS, IAM, CloudWatch,Cloudformation, EventBridge) as part of cloud-native data pipelines.
4. Code reusable frameworks, utilities, and components to accelerate team productivity.
5. Conduct peer code reviews, mentor team members, and establish coding standards.
Quality & Delivery
1. Perform unit testing and support integration/UAT activities.
2. Optimize workflows for performance, cost-efficiency, and maintainability.
3. Ensure version control, CI/CD integration, and documentation are consistently maintained.
4. GitHub CI/CD integration
⦁ Production Support & Issue Resolution Responsibilities
Incident & Problem Management
1. Serve as the point for any production issues across DataStage, SnapLogic, Snowflake, AWS, and Tidal.
2. Responsible for analyzing and fixing production issues in any system.
3. Analyze root causes, identify systemic issues, and propose permanent fixes.
4. Develop automated monitoring, alerting, using Python/AWS.
Stability & Reliability
1. Work closely with operations teams to ensure SLAs, uptime, and performance thresholds are met.
2. Improve recovery processes, failover mechanisms, and operational resiliency.
3. Troubleshoot orchestration and scheduling issues in Tidal Enterprise Scheduler.
⦁ Architecture Input & Technical Leadership Responsibilities
Architecture Contributions
1. Provide architectural guidance for new integrations, data warehouse initiatives, and migration projects.
2. Recommend best practices for cloud adoption, pipeline modernization, and platform evolution.
3. Contribute to reference architectures, standards, reusable patterns, and decision frameworks.
Strategic Technology Guidance
1. Evaluate emerging tools and technologies in data engineering, integration, cloud, and orchestration.
2. Champion automation, observability, data quality, and performance optimization initiatives.
3. Collaborate with enterprise architects and infrastructure teams to ensure future-proof design choices.
⦁ Cross-functional Collaboration Responsibilities
1. Work with QA, business analysts, and project managers to enable smooth delivery.
2. Assist and collaborate with offshore teams to ensure alignment, effective knowledge transfer, and smooth delivery across time zones.
3. Communicate technical concepts clearly to both technical and non-technical stakeholders.
⦁ Tools & Technology Scope
1. ETL/ELT: IBM DataStage, SnapLogic
2. Cloud: AWS (Compute, Storage, Serverless, Orchestration, IAM, Monitoring)
3. Data Warehouse: Snowflake
4. Programming: Python, SQL
5. Scheduling: Tidal Enterprise Scheduler
6. Other: Git, CI/CD tools, DevOps pipelines, monitoring platforms
Minimum Years of Experience:
⦁ 10 years
Work Location & Reporting Address: Peoria, IL 61630
Contract duration: 12
Must Have Skills:
⦁ SnapLogic
⦁ AWS
⦁ Python
⦁ Snowflake
⦁ DataStage
Nice to Have Skills:
⦁ GitHub CI/CD integration
Detailed Job Description:
⦁ Design Responsibilities
Solution & Technical Design
1. Translate business requirements into scalable, maintainable, and secure technical designs across DataStage, SnapLogic, AWS, Python, and Snowflake ecosystems.
2. Create high-level and low-level design documents covering data flows, mappings, process orchestration, and integration patterns.
3. Develop data models, transformation logic, and interface specifications for batch, real-time, and event-driven pipelines.
4. Ensure designs align with enterprise architecture standards, security guidelines, and best practices.
5. Collaborate with product owners, analysts, and architects to finalize solution design and validate technical feasibility.
⦁ Senior Development Responsibilities
Hands-on Development & Implementation
1. Build and enhance ETL/ELT pipelines in IBM DataStage, SnapLogic, and Python.
2. Develop scalable Snowflake SQL, stored procedures, and performance-optimized ELT logic.
3. Implement AWS services (Lambda, S3, SNS, SQS, IAM, CloudWatch,Cloudformation, EventBridge) as part of cloud-native data pipelines.
4. Code reusable frameworks, utilities, and components to accelerate team productivity.
5. Conduct peer code reviews, mentor team members, and establish coding standards.
Quality & Delivery
1. Perform unit testing and support integration/UAT activities.
2. Optimize workflows for performance, cost-efficiency, and maintainability.
3. Ensure version control, CI/CD integration, and documentation are consistently maintained.
4. GitHub CI/CD integration
⦁ Production Support & Issue Resolution Responsibilities
Incident & Problem Management
1. Serve as the point for any production issues across DataStage, SnapLogic, Snowflake, AWS, and Tidal.
2. Responsible for analyzing and fixing production issues in any system.
3. Analyze root causes, identify systemic issues, and propose permanent fixes.
4. Develop automated monitoring, alerting, using Python/AWS.
Stability & Reliability
1. Work closely with operations teams to ensure SLAs, uptime, and performance thresholds are met.
2. Improve recovery processes, failover mechanisms, and operational resiliency.
3. Troubleshoot orchestration and scheduling issues in Tidal Enterprise Scheduler.
⦁ Architecture Input & Technical Leadership Responsibilities
Architecture Contributions
1. Provide architectural guidance for new integrations, data warehouse initiatives, and migration projects.
2. Recommend best practices for cloud adoption, pipeline modernization, and platform evolution.
3. Contribute to reference architectures, standards, reusable patterns, and decision frameworks.
Strategic Technology Guidance
1. Evaluate emerging tools and technologies in data engineering, integration, cloud, and orchestration.
2. Champion automation, observability, data quality, and performance optimization initiatives.
3. Collaborate with enterprise architects and infrastructure teams to ensure future-proof design choices.
⦁ Cross-functional Collaboration Responsibilities
1. Work with QA, business analysts, and project managers to enable smooth delivery.
2. Assist and collaborate with offshore teams to ensure alignment, effective knowledge transfer, and smooth delivery across time zones.
3. Communicate technical concepts clearly to both technical and non-technical stakeholders.
⦁ Tools & Technology Scope
1. ETL/ELT: IBM DataStage, SnapLogic
2. Cloud: AWS (Compute, Storage, Serverless, Orchestration, IAM, Monitoring)
3. Data Warehouse: Snowflake
4. Programming: Python, SQL
5. Scheduling: Tidal Enterprise Scheduler
6. Other: Git, CI/CD tools, DevOps pipelines, monitoring platforms
Minimum Years of Experience:
⦁ 10 years
Best Regards,
|
Salary : $48 - $50