Company DescriptionMajor CorporationJob DescriptionResponsibilities:Design and develop high-throughput, low-latency data processing pipelines to quickly ingest and make data available on the platform across various distributed data storesAnalyze large datasets to identify opportunities to tune and improve the systemExperiment with various Hadoop frameworks like Hive, Pig and Scalding to identify the optimal approach for extracting valuable insights from massive datasetsTool We Used:ScalaHadoop (Hive, Pig, Scalding, Spark)KafkaMySQL, Redis, Vertica, AerospikeQualificationsRequirements:Bachelors or Masters in Computer Science or related field3 years of experience ingesting, processing, storing and querying large datasetsProfessional Hadoop ecosystem experience, including storage optimization and job performance tuningExpertise in Java, Python or similar language(s). Functional programming experience is a plusPassion for code correctness and intuition about which values in data are to be expected in a business contextAdditional InformationAll your information will be kept confidential according to EEO guidelines.