What are the responsibilities and job description for the Cloud Data Solutions Engineer position at Resourcesoft, Inc.?
Requirements
-
8 or more years of experience in Big Data architecture and distributed data warehousing solutions.
-
Proficiency in Spark and Scala for high-performance data processing and framework development.
-
Experience with Kafka and Aerospike for real-time data streaming and low-latency storage.
-
Experience in AWS cloud infrastructure including EKS, EMR, Redshift, Athena, and S3.
-
Experience with Java and Python programming for building scalable data pipelines.
-
Experience in Hadoop ecosystems, SQL-based data manipulation, and Talend integration tools.
-
Excellent verbal and written communication skills.
Responsibilities
-
Architect and develop scalable Big Data warehouse solutions across the entire data supply chain.
-
Lead the design and implementation of metadata management systems and architectural diagrams.
-
Expand and enhance enterprise Data Lake environments to support business-driven decisioning.
-
Integrate complex data sets across multiple systems using real-time analysis strategies.
-
Partner with Data Science teams to optimize actionable data and analytical toolsets.
-
Automate data delivery pipelines using Jenkins CI/CD and Git-based version control.
-
Implement Data Governance and Security protocols within distributed cloud environments.