What are the responsibilities and job description for the Big Data Engineer Python/Scala, Spark, Hadoop (LOCAL CANDIDATES ONLY) position at SumasEdge Corporation?
Job Summary:
- Ability to easily move between business, data management, and technical teams; ability to quickly intuit the business use case and identify technical solutions to enable it
- Experience building robust and efficient data pipelines end-to-end with a strong focus on data quality
- High proficiency in using Python or Scala, Spark, Hadoop platforms & tools (Hive, Airflow, NiFi, Scoop), SQL to build Big Data products & platforms
- Experience in building and deploying production-level data-driven applications and data processing workflows/pipelines and/or
- Implementing machine learning systems at scale in Java, Scala, or Python and deliver analytics involving all phases like data ingestion, feature engineering, modeling, tuning, evaluating, monitoring, and presenting
- Cloud knowledge (Databricks or AWS ecosystem) is a plus but not required