What are the responsibilities and job description for the Senior Data Engineer position at ARK Infotech Spectrum?
Role : Senior Data Engineer
Location : Tampa,FL
Local candidates only. (Hybrid Onsite)
This is a 3-month contract to hire position.
Candidates who are authorized to work for any employer in US without requiring visa sponsorship now or in future can apply
Job Duties:
Given the input from the Data Scientist and Data Modeler, create the data pipeline to ingest, cleanse and standardize the data so that it can be ready for analysis.
Once the data is processed, apply the machine learning algorithm as prescribed by the Data Scientist with the production performance readiness. Finally, once the result is processed, activate the models through either Micro Services, Rporting, Dashboard, or emails
• Experience with Big Data Technologies or Cloud Technologies building data pipelines
• DevOPs Experience (source control, continuous integration, deployments, etc.)
• Experience with containerization and related technologies (e.g. Docker, Kubernetes)
• Experience in other open-sources like Druid, Elastic Search, Logstash etc
• Advanced knowledge of the Hadoop ecosystem and Big Data technologies
• Hands-on experience with the Hadoop eco-system (HDFS, MapReduce, Hive, Pig, Impala, Spark, Kafka, Kudu, Solr)
• System level understanding - Data structures, algorithms, distributed storage & compute
• Exposure and / or development experience in Microservices Architectures best practices
• Experience around REST APIs, services, and API authentication schemes
• Knowledge in RDBMS and NoSQL technologies
Minimum Skills Required:
10 years of the following experience:
• Experience with Big Data Technologies or Cloud Technologies building data
pipelines
• DevOPs Experience (source control, continuous integration, deployments, etc.)
• Experience with containerization and related technologies (e.g. Docker, Kubernetes)
• Hands-on experience with the Hadoop eco-system (HDFS, MapReduce, Hive, Pig, Impala, Spark, Kafka, Kudu, Solr)?