What are the responsibilities and job description for the Data Engineer - Python, Spark, hadoop position at SRI Tech Solutions?
Job Description:
- Ability to easily move between business, data management, and technical teams; ability to quickly understand the business use case and identify technical solutions to enable it
- Experience building robust and efficient data pipelines end-to-end with a strong focus on data quality
- High proficiency in using Python or Scala, Spark, Hadoop platforms & tools (Hive, Airflow, NiFi, Sqoop), SQL to build Big Data products & platforms
- Experience in building and deploying production-level data-driven applications and data processing workflows/pipelines
- Implementing machine learning systems at scale in Java, Scala, or Python and delivering analytics involving all phases like data ingestion, feature engineering, modeling, tuning, evaluating, monitoring, and presenting
- Cloud knowledge (Databricks or AWS ecosystem) is a plus but not required