Key Skills: AWS, S3, RedShift, Glue, PySpark, SQL

JD/Description of Role*

Experience in building data solutions with AWS technologies: storage, pipelines, ingestion, job scheduling, notifications /alerts, logging

Experience in data processing: full/incremental, batch/real-time/near-real-time, event driven

Experience building data solutions using AWS services such as Redshift, RDS, MWAA, Glue, S3, SQS, SNS

Proficiency in Python 3.6 , PySpark, SQL

Proficiency in Databricks and Medallion architecture on AWS platform

Implementing jobs monitoring with Amazon CloudWatch

Experience deploying applications using Terraform or a similar IaC tool

Experience building CI/CD pipelines using GitHub Actions or a similar tool

Proficiency in managing code as part of a team using GitHub

Familiarity with agile development methodologies

Experience building data-driven unit test suites for data platforms

Experience with Domain Driven Design for data modelling

Good to have

Experience in the biotechnology /biopharmaceutical industry

Knowledge of SRE/ Observability practices

Architecting high performant and scalable data engineering solution with Databricks on AWS data platform

Architecting data solutions in Medallion data architecture definition and process for a Data Lake, Lakehouse, DW, analytical marts

Proficiency in data mesh architecture; data domain driven distributed /decentralized data solution, governance for data federation

Architect and design data pipeline patterns with aspects of process audit, data quality, cost optimized design

Perform data analysis and define data models

Setup connections and structure for the development components

Knowledge of data governance and security solutions with regulatory compliance requirements

Primary (Must have skills)*

AWS, S3, RedShift, Glue, PySpark, SQL

Secondary Skills (Good To have)*

Remaining

Soft skills/other skills (If any)

Should have strong technical experience to architect/design AND good communication and collaborative skills to work with the clients effectively

Companies- Associated with