What are the responsibilities and job description for the Senior Data Engineer – Life Sciences / Pharma position at Value Technology Inc?
Job Title: Senior Data Engineer – Life Sciences / Pharma
Location: South San Francisco, CA (Hybrid – 3 Days Onsite)
Duration: 8–12 Months Contract
Job Description
Value Technology is seeking a highly experienced Senior Data Engineer with a strong background in Life Sciences and Pharmaceutical industries to join a long-term strategic data transformation initiative. The ideal candidate will possess deep expertise in AWS-based Data Engineering technologies, data integration, large-scale ETL/ELT pipeline development, and enterprise data platforms.
The candidate should have hands-on experience building scalable and reliable data solutions for clinical, research, commercial, or healthcare-related datasets while ensuring data quality, governance, and compliance standards.
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT pipelines using AWS cloud technologies.
- Build enterprise-grade data ingestion and integration frameworks for structured and semi-structured datasets.
- Develop and optimize PySpark and SQL-based transformations for large-scale data processing.
- Implement data orchestration workflows using Apache Airflow for batch and real-time processing pipelines.
- Design and maintain scalable data lake and data warehouse solutions using AWS S3, Redshift, RDS, and EMR.
- Develop data quality, profiling, validation, and reconciliation frameworks to ensure data integrity and compliance.
- Collaborate with business stakeholders, data scientists, and analytics teams to deliver data products and reporting solutions.
- Implement API-based integrations using AWS API Gateway and Lambda services.
- Design and maintain dimensional data models including Star Schema and Snowflake Schema for analytical reporting.
- Perform performance tuning and optimization of ETL pipelines, Spark jobs, and SQL queries.
- Develop reusable data engineering frameworks and metadata-driven processing solutions.
- Ensure adherence to data governance, security, and compliance standards within Life Sciences and Pharma environments.
- Work closely with cross-functional teams in Agile/Scrum environments for sprint planning and delivery.
- Support cloud migration and modernization initiatives from legacy platforms to AWS cloud environments.
- Monitor, troubleshoot, and resolve production pipeline failures and performance bottlenecks.
- Build scalable data products supporting research, clinical, operational, and commercial analytics initiatives.
Required Skills
- 10 years of experience in Data Engineering and Data Integration
- Strong Life Sciences / Pharmaceutical industry experience is mandatory
- Expertise in ETL/ELT pipeline development and orchestration
- Strong experience with Data Modeling and Data Warehousing concepts
- Experience with Data Profiling, Data Quality, and Data Validation frameworks
- Experience building scalable cloud-native data platforms
- Knowledge of Data Products and modern data architecture concepts
- Experience with CI/CD, Git, and Agile methodologies
Preferred Qualifications
- Experience working with clinical, healthcare, laboratory, or pharmaceutical datasets
- Experience with real-time data streaming and event-driven architectures
- Familiarity with HIPAA, GxP, FDA, or healthcare compliance standards
- Exposure to Databricks or Snowflake is a plus
- Strong analytical, communication, and stakeholder management skills