What are the responsibilities and job description for the Senior AWS Data Engineer position at Baanyan Software Services, Inc.?
Title: Senior AWS Data Engineer
Location: Jersey City, NJ (5 days onsite)
Duration: Long term
Experience: 8 yrs minimum
Job Description:
We are seeking an experienced Senior Engineer with strong programming and cloud platform expertise. The ideal candidate will have hands-on experience across core AWS services, a solid background in Python, PySpark, SQL, and data modeling, along with proficiency in leveraging AI development tools (e.g., GitHub Copilot) to enhance engineering productivity.
Key Responsibilities:
- Write efficient, maintainable code in Python, leveraging frameworks like PySpark for large-scale data processing.
- Investigate and troubleshoot data pipelines, demonstrating a strong understanding of data structures and distributed processing.
- Develop and optimize complex SQL queries; implement data warehousing concepts including Slowly Changing Dimensions (SCD Type 1 & Type 2).
- Design, develop, and deploy scalable, reliable APIs on AWS (ECS).
- Work extensively with core AWS services, including ECS, Lambda, SNS, and DynamoDB.
- Build, test, and deploy applications via CI/CD pipelines to ensure automated delivery.
- Utilize AI-powered developer tools (such as GitHub Copilot) to accelerate delivery and code quality.
- Collaborate with cross-functional teams to troubleshoot and resolve cloud infrastructure and application performance issues.
Required Skills & Qualifications:
- AWS Services: Hands-on experience with AWS ECS, Lambda, SNS, and DynamoDB.
- Programming & Data: Proficient in Python with strong PySpark experience for data pipeline investigation and processing.
- Databases & Modeling: Solid understanding of SQL, data modeling concepts, and SCD (Type 1 & Type 2).
- API & Cloud Deployment: Proven experience building and deploying APIs on AWS / ECS.
- DevOps: Working knowledge of CI/CD processes and pipelines.
- AI Tooling: Experience using AI-assisted coding tools (GitHub Copilot, etc.).
Strong analytical, debugging, and problem-solving skills.