What are the responsibilities and job description for the SAP Qualtrics Data Integration Engineer position at Confidential Jobs?
AWS Data Engineer – Qualtrics Integration
Role Type: Contract
Location: Torrance, CA / Remote
Domain: Enterprise Survey Platform / Qualtrics
Primary Stack: AWS | Python | SQL | Qualtrics API
The AWS Data Engineer – Qualtrics Integration is responsible for designing, building, and maintaining scalable, automated data pipelines that support Qualtrics survey ingestion, transformation, and downstream reporting.
This role focuses on serverless AWS data engineering, integrating Qualtrics APIs with AWS services to process structured survey data, dealer hierarchies, and reporting files. The engineer will ensure data accuracy, automation, monitoring, and performance across end-to-end workflows.
Key Responsibilities
AWS Cloud & Data Engineering
- Design and maintain ETL pipelines using AWS Glue (PySpark)
- Develop AWS Lambda functions (Python / Node.js) for serverless data processing
- Manage AWS S3 for Qualtrics input/output storage and optimized data access
- Orchestrate workflows using AWS Step Functions and MWAA (Airflow)
- Query structured datasets using AWS Athena
- Monitor pipelines using CloudWatch logs and metrics
Data Processing & ETL
- Transform and aggregate Qualtrics datasets (CSV, JSON, XML)
- Merge multiple source files (e.g., dealer master employee files) into unified hierarchies
- Automate ingestion, transformation, validation, and report generation
- Implement reusable, scalable ETL frameworks
API & Integrations
- Integrate with Qualtrics APIs to extract raw survey data and response files
- Implement event-driven processing (S3 triggers → Lambda)
- Connect AWS pipelines with CRM, ERP, BI tools, or downstream platforms
Data Quality & Reporting
- Perform data validation and quality checks prior to Qualtrics ingestion
- Generate formatted output files aligned to business-defined templates
- Support ad-hoc analysis using SQL and Athena
Programming & Automation
- Develop robust Python scripts for Glue, Lambda, and automation tasks
- Write optimized SQL queries for structured data access
- Use Bash/Shell scripting for file movement and preparation
DevOps & Security (Nice to Have)
- Configure IAM roles and permissions securely
- Implement Infrastructure as Code (Terraform / CloudFormation)
- Support CI/CD pipelines for data workflows
Required Skills & Experience
- Strong experience in AWS Data Engineering & Serverless Architecture
- Hands-on expertise with AWS Glue, Lambda, S3, Athena, Step Functions
- Experience with MWAA (Airflow) for orchestration
- Strong Python and SQL skills
- Experience integrating Qualtrics API or structured survey data
- Ability to troubleshoot pipeline failures and performance issues
Preferred / Nice-to-Have Skills
- Experience with Terraform or CloudFormation
- CI/CD for data pipelines
- Experience supporting BI tools (Power BI, Tableau, etc.)
- Knowledge of data governance and security best practices
Ideal Candidate Profile
- AWS-focused Data Engineer with strong automation mindset
- Comfortable working with survey / VoC / structured data
- Strong debugging, monitoring, and optimization skills
- Able to work independently in enterprise environments