What are the responsibilities and job description for the Talend Architect position at Net2Source (N2S)?
Title- Data Engineer
Location- Reading, PA, Onsite
The Role
As a Senior Data Engineer, you will design, build, and maintain end-to-end data pipelines leveraging Talend Cloud (8.0), PySpark, and AWS services. You will play a key role in ingesting, transforming, and optimizing large-scale structured and unstructured datasets while ensuring scalability, performance, and data quality across the platform.
Key responsibilities include:
- Designing and developing ETL/ELT workflows using Talend 8.0 on Cloud
- Integrating data from APIs, flat files, and streaming sources
- Ingesting and managing data in AWS S3–based data lakes
- Developing PySpark jobs for large-scale data processing and transformations
- Implementing Spark SQL for complex transformations and schema management
- Building and supporting cloud-native data pipelines using AWS services such as Glue, Lambda, Athena, and EMR
- Applying performance tuning and optimization techniques for Spark workloads