Job DescriptionHello,Hope you are doing well.Job Title:Big data engineerDuration: 6 MonthsLOCATION: 1st Preference Iselin or Charlotte ONSITE And New York, NJOnly GC/CitizensJob Description:Scope of WorkPOSITIONAs a Lead Engineer you will have the opportunity to engineer and administer TIAA’s big data environment. Your role will be responsible for administering our Hadoop and No-SQL ecosystem components such as HDFS, Hive, MR, Yarn, Impala, Spark, Sqoop, HBase, Sentry, Hue and Oozie. Your role will design and implement automated processes, research database technologies, communicate effectively with database administrators and application stake holders to ensure your internal clients’ needs are met.RESPONSIBILITIESResponsible for the implementation and on-going administration of Hadoop infrastructure including the installation, configuration and upgrading of Cloudera distribution of HadoopFile system, cluster monitoring, and performance tuning of Hadoop ecosystemResolve issues involving map reduce, yarn, sqoop job failures; Analyze multi-tenancy job execution issues and resolveDesign and manage backup and disaster recovery solution for Hadoop clustersWork on Unix operating systems to efficiently handle system administration tasks related to Hadoop clustersManage the Apache Kafka and Apache NIFI environmentsParticipate and manage the data lakes data movements involving Hadoop, NO-SQL databases like HBase, Cassandra and MongodbWork with data delivery teams to setup new Hadoop users. Includes setting up Linux users, setting up Kerberos principals and testing HDFS, Hive, Pig and Map Reduce access for the new users. Configure Hadoop security aspects including Kerberos setup and RBAC authorization using Apache SentryCreate and document best practices for Hadoop and big data environmentParticipate in new data product or new technology evaluations; manage the certification process and evaluate and implement new initiatives in technology and process improvementsInteract with Security Engineering to design solutions, tools, testing and validation for controlsEvaluate the database administration and operational practices, and evolve automation procedures (Using scripting languages such as Shell, Python, Chef, Puppet, CFEngine, Ruby etc.)Advance the cloud architecture for data stores; Work with TIAA Cloud engineering team with automation; Help operationalize Cloud usage for databases and for the Hadoop platformEngage vendors for feasibility of new tools, concepts and features, understand their pros and cons and prepare the team for rolloutAnalyze vendor suggestions/recommendations for applicability to TIAA’s environment and design implementation detailsPerform short and long term system/database planning and analysis as well as capacity planningIntegrate/collaborate with application development and support teams on various IT projectsQUALIFICATIONSRequired ExperienceBachelor’s degree; Preferably in Computer Science or Information SystemsTen or more years of overall IT/DBMS/Data Store experienceThree or more years of experience in, big data, data caching, data federation and data virtualization management including experience in leveraging HadoopTwo or more years of expertise and in-depth knowledge of SAN, system administration, VmWare, backups, restores, data partitioning, database clustering and performance managementExperience writing shell scripts, and automating tasks. Exposure to Chef or/and Puppet is preferredExperience in the implementation details of Hadoop Clusters, Impala, and HBase and other emerging data techniquesExperience with monitoring technologies for databasesExperience with orchestration techniques, infrastructure automation and cloud deploymentsUnderstating of Linux, Windows, Dockers / containersFamiliarity with “IaaS” and “DBaaS” Service oriented concepts preferredFamiliarity of Cloud Architecture (Public and Private clouds) – AWS , AZURE preferredWorking knowledge of VMware and VMware vCloud Automation Center (vCAC) preferredDesired ExperienceProficiency in using Microsoft Office (Word, Excel, PowerPoint) to document, present, communicate and articulate idea/s and conceptsStrong communication skills and the ability to collaborate and work in teams with other engineers, working in a fast paced and ever changing technical environmentApplication development experience – database programming, scripting, setting up web sites and dashboardsAdditional InformationAll your information will be kept confidential according to EEO guidelines.