What are the responsibilities and job description for the Senior Data Engineer (Google Cloud Platform, Spark, Python) position at Euclid Innovations?
Job Description:
We are looking for a Senior Data Engineer with strong experience in Google Cloud Platform, Spark, and Python to support enterprise data engineering and cloud migration initiatives.
Required Skills:
- 12 years of IT experience
- Strong hands-on experience with Spark and Python (Mandatory)
- Experience migrating data from on-premises to Google Cloud Platform
- Strong experience building batch and streaming data pipelines
- Hands-on experience with BigQuery, Cloud Storage, Dataplex, Dataflow, Dataproc, Pub/Sub, Cloud Composer, and Vertex AI
- Experience with HDFS, NFS, SQL Server, and object storage
- Strong understanding of data quality, metadata, schema management, and data lineage
- Experience supporting ML training and inference workloads, feature engineering, and feature stores
- Excellent communication and problem-solving skills