What are the responsibilities and job description for the Senior Data Engineer position at Tredence Inc.?
Data Engineer (GCP & Data Modelling)
Position Summary
We are seeking a highly skilled Data Engineer with strong expertise in Data Modelling, Python/PySpark, and hands-on experience across Google Cloud Platform (GCP) data services. The ideal candidate will design, build, and optimize scalable data pipelines while ensuring data quality, reliability, and performance for analytics and business insights.
Key Responsibilities
Data Engineering & Modelling
- Design, develop, and maintain data models (conceptual, logical, and physical) to support analytical and operational use cases.
- Build scalable ETL/ELT pipelines using Python, PySpark and SQL.
- Optimize data workflows for performance, cost efficiency, and reliability.
Google Cloud Platform (GCP)
- Develop and manage data solutions using:
- BigQuery (data warehousing, query optimization, partitioning, clustering)
- Cloud Storage (data ingestion, file management, lifecycle optimization)
- DataProc (Spark/Hadoop clusters, distributed data processing)
- DataStore / Firestore (NoSQL database design & integration)
- Implement CI/CD pipelines and automation using GCP DevOps tools (Cloud Build, Cloud Composer, Cloud Functions, etc.)
Programming & Data Processing
- Write clean, maintainable code using Python and PySpark.
- Work with large structured and unstructured datasets.
- Conduct performance tuning of SQL queries and Spark jobs.
Data Quality & Governance
- Ensure data accuracy, completeness, and consistency across systems.
- Implement data validation, monitoring, and logging frameworks.
Collaboration
- Work closely with data scientists, analysts, architects, and business partners to translate requirements into scalable data solutions.
- Contribute to documentation, best practices, and knowledge sharing within the team.
Required Skills & Qualifications
- Bachelor’s or Master’s degree in Computer Science, Engineering, Information Systems, or equivalent experience.
- 8 years of experience in Data Engineering
- Strong expertise in:
- Data Modelling
- Python & PySpark
- SQL (advanced level)
- Hands-on experience with key GCP services:
- BigQuery
- Cloud Storage
- DataProc
- DataStore / Firestore
- GCP DevOps tools & CI/CD
- Experience with distributed data processing and large-scale datasets.
- Strong analytical and problem-solving skills.
Preferred Qualifications
- Experience with workflow orchestration tools (e.g., Cloud Composer, Airflow).
- Knowledge of data governance, cataloging, and metadata management.
- Familiarity with containerization (Docker, Kubernetes).
Why Join Us?
- Opportunity to work with modern cloud and big-data technologies.
- Collaborative environment with exposure to cross-functional teams.
- Ability to influence data architecture and engineering best practices.