Ab Initio Data Engineer

Dear Applicant,

Please let me know if you are interested.

Position: Ab Initio Data Engineer

Location: Berkeley heights, NJ (5 days onsite per week)

Hire Type: long term Contract

Job Description:

Job Description:

We are seeking a highly skilled resource to design and implement high-performance, event-driven data pipelines, ensuring low-latency data processing and high availability system for the large credit card processing system. The ideal candidate will work with the Ab Initio ecosystem (GDE, EME, Conduct>It) to build stateful services that ingest, filter, and transform data from sources like Kafka or message queues, pushing updates to dashboards or downstream databases in near-real-time.

Responsibilities

• Design, develop, and maintain complex ETL graphs using Ab Initio GDE (Graphical Development Environment)

• Architect end-to-end data integration solutions spanning multiple source/target systems (RDBMS, mainframe, flat files, message queues)

• Optimize graph performance — parallelism, partitioning (broadcast, hash, range), layout tuning, checkpointing

• Lead development using Ab Initio components: Co>Operating System, EME (Enterprise Meta>Environment), Conduct>It, Data Profiler, Continuous Flows

• Own metadata management and data lineage within EME

• Design and implement CDC (Change Data Capture) and near-real-time data pipelines

• Troubleshoot production job failures, perform root-cause analysis, and implement fixes with minimal downtime

• Define coding standards, review peer-developed graphs, and enforce best practices

• Mentor junior/mid-level Ab Initio developers

• Collaborate with data architects, DBAs, and business analysts to translate requirements into technical design

Required Qualifications

• Bachelor’s degree in Computer Science, Engineering, or related field

• 7 years of ETL development experience, with 5 years specifically in Ab Initio

• Strong hands-on expertise with Ab Initio GDE, Co>Op, EME, and Conduct>It

• Deep understanding of parallel processing concepts and Ab Initio partitioning/multi-file system (MFS)

• Strong SQL skills and experience with major RDBMS (DB2)

• Experience with Unix/Linux shell scripting

• Solid understanding of data warehousing concepts (dimensional modeling, SCDs, star/snowflake schemas)

• Experience with Kafka

• Strong debugging and performance-tuning skills for large-volume data processing

Preferred Qualifications

• Experience with mainframe data integration (COBOL copybooks, VSAM, flat files)

• Exposure to Kubernetes, openshift

• Familiarity with Hadoop/Spark ecosystems and hybrid ETL architectures

• Knowledge of API-based or streaming integration (Kafka)

• Prior experience as a technical lead or in a client-facing architecture role

Soft Skills / Leadership

• Strong analytical and problem-solving skills for complex data issues

• Ability to lead design discussions and drive technical consensus

• Effective mentorship and knowledge-sharing across the team

• Clear communication with both technical and business stakeholders

Behavioral Skills:

1. Good Communication skills
2. 5 days Work from Office at Berkley Heights, NJ
3. Team Player
4. Ability to work in a changing environment
5. Strong problem solving and analytical skills
6. Ability to work independently or within a team
7. Manage day-to-day challenges and communicate developmental risks with the technical team