What are the responsibilities and job description for the Data Engineeer - W2 only position at Innovative IT Solutions Inc?
Hi
Must have and focus will be on python, sagemaker, spark, CICD with jenkins
Candidates must take a test
Data Engineeer - Vanguard
Onsite 3 days a week in Malvern PA.
Rate:45/h on wob
Role Overview:
We are seeking a mid-level Data Engineer (3–5 years of experience) to join our Brokerage and Investment team. You will act as a senior engineer, engaging directly with business analysts and stakeholders to translate complex requirements into robust, global data solutions. This is a front-office role, so excellent communication skills and the ability to work independently with minimal hand-holding are essential.
Technical Stack & Requirements:
- Core Competencies: AWS, Glue, S3, IAM, Redshift, Python, Spark.
- Development Environment: Experience with Amazon SageMaker is mandatory, as this is our primary environment for code development before production deployment.
- Pipeline Management: Experience building end-to-end data pipelines, managing SFTP connections, and integrating with diverse data sources (Data Lakes, flat files, etc.).
- Infrastructure & Automation: Experience with SNS notifications and CI/CD tools (Jenkins) to ensure process efficiency.
- Nice-to-Have Skills: Experience with Databricks or Snowflake is highly desirable as we transition toward these platforms.
- Data Modeling: Proficiency in designing data models and creating multi-layered data structures to feed executive and operational reports.
Key Responsibilities:
- Manage priorities across multiple workstreams and handle production support as needed.
- Maintain robust pipelines with automated checks, particularly for event-driven jobs.
- Communicate effectively with external vendors and internal business stakeholders to gather requirements and provide solutions.
- Collaborate with senior engineering staff to ensure the security and integrity of our AWS development environments.
Required Qualifications
- 5 years of Data Engineering or IT experience.
- 4 years of AWS cloud development experience.
- Expert Python programming skills.
- Experience with Sagemaker.
- Robust hands-on experience with Apache Spark / PySpark.
- Experience building cloud-native ETL pipelines.
- Robust understanding of distributed computing.
- Experience designing scalable data models.
- Experience with Git and CI/CD pipelines.
- Robust analytical and problem-solving skills.
- Build scalable and auto recoverable pipelines.
- Bachelor''s degree in Computer Science, Engineering, Information Systems, or related discipline.
- Working directly with business SME’s and stakeholders.
Good to have
- Experience with Databricks tool.
Salary : $35 - $45