Data Engineer with Migration

About Us:

ITC Infotech is a leading global technology services and solutions provider, led by Business and Technology Consulting. ITC Infotech provides business-friendly solutions to help clients succeed and be future-ready, by seamlessly bringing together digital expertise, strong industry specific alliances and the unique ability to leverage deep domain expertise from ITC Group businesses. We provide technology solutions and services to enterprises across industries such as Banking & Financial Services, Healthcare, Manufacturing, Consumer Goods, Travel and Hospitality, through a combination of traditional and newer business models, as a long-term sustainable partner.

Role Overview

We are looking for a skilled Data Engineer to join the Datastore Migration Factory team, responsible for executing end-to-end datastore migration from on-prem Data Lake to an AWS-hosted Lakehouse environment.

This is a high-visibility and business-critical initiative, requiring strong technical expertise, stakeholder management, and a focus on data integrity and quality.

Key Responsibilities

1. Pipeline Migration

• Refactor and migrate data extraction logic and job scheduling from legacy systems to the Lakehouse environment
• Execute data migration while ensuring data integrity and consistency
• Collaborate with stakeholders to validate migrated datasets and complete hand-off/sign-off processes

2. Consumption Pattern Migration

• Convert and optimize legacy SQL and Spark-based workloads for Snowflake and Apache Iceberg
• Analyze usage patterns to deliver optimized and scalable data solutions
• Engage with stakeholders to ensure business requirements are met post-migration

3. Data Reconciliation & Quality Assurance

• Perform rigorous data validation and reconciliation
• Work with reconciliation frameworks to ensure migrated data matches production systems
• Maintain high standards for data accuracy, completeness, and reliability

4. Collaboration & Platform Integration

• Work closely with internal data platform and engineering teams
• Adapt to new tools, workflows, and technologies as required

Technical Skills

Basic Qualifications

• Bachelor’s or Master’s degree in Computer Science, Engineering, Applied Mathematics, or a related field
• 7 years of hands-on development experience in a collaborative environment
• Strong troubleshooting skills (especially in SQL) and scripting experience
• Proficiency in Python or Java
• Good understanding of SDLC, CI/CD practices, and Kubernetes (K8s) deployments

Core Data Engineering Competencies

• Temporal Data Modeling (e.g., SCD Type 2)
• Schema Management (Schema Evolution – Apache Iceberg)
• Performance Optimization (partitioning, clustering)
• Data Architecture (Normalization vs. Denormalization, Natural vs. Surrogate Keys)

Technical Stack

• Data Processing & Extraction: Kafka, ANSI SQL, FTP, Apache Spark
• Data Formats: JSON, Avro, Parquet
• Platforms: Hadoop (HDFS/Hive), Snowflake, Apache Iceberg, Sybase IQ

ITC Infotech is an Equal Opportunity Employer. We believe that no one should be discriminated against because of their differences, such as age, disability, ethnicity, gender, gender identity and expression, religion, or sexual orientation. All employment decisions shall be made without regard to age, race, creed, color, religion, sex, national origin, ancestry, disability status, veteran status, sexual orientation, gender identity or expression, genetic information, marital status, citizenship status or any other basis as protected by federal, state, or local law. ITC infotech is committed to providing veteran employment opportunities to our service men and women.