What are the responsibilities and job description for the HPC Dataflow Analyst position at Axiom Consultants?
Axiom is seeking a talented HPC Dataflow Analyst to join the team. This individual will primarily be responsible for forecast products and model guidance and will work closely with the IDP team on all data dissemination within the NCO environment.
NCEP Central Operations (NCO) provides and exchanges over 12TB per day of all different data types and formats with the NWS offices and centers, core Government partners, the education sector, and international partners to support the mission. The data types consist of Watches/Warnings and Advisories (WWAs), forecast products, model guidance, observational, radar, and satellite data, which are freely provided to the general public via web services. The inbound and outbound data transmissions and processing are supported by two Dataflow Teams within NCO: IDP Dataflow and HPC Dataflow.
Responsibilities:
-
Provide tier 2 O&M support during normal business hours and off-hours in an on-call rotation for all data flow systems by: retrieving, disseminating, ingesting, processing, testing, archiving, verifying, optimizing, troubleshooting, timing, reporting, and monitoring data feeds.
-
For all new data related requests, identify user requirements, write detailed implementation instructions, test changes in development whenever possible, communicate the change timelines with all users (requestor and/or Government contracted vendor partners), successfully execute NCO approved instructions in operations following all NCO CM policies, and verify all changes worked according to requirements.
-
Provide guidance, optimizations, and recommendations, to all government contracted vendor partners with respect to data flowing between systems.
-
Provide thorough end-to-end data checkouts for implementations and/or upgrades of applications/models by following the established standard procedures.
-
Support all software used to transfer data products by being able to: update, modify, test, troubleshoot, and optimize code and scripts written in languages such as: Perl, Python, C, and bash.
-
Be experienced with data transfer protocols on Linux based systems to transfer data in the most efficient methods using: Local Data Manager (LDM), File Transfer Protocol (FTP), FTP Secure (FTPs), Secure Copy (SCP), and Remote Sync (RSync). In addition, must have knowledge and experience transferring data in and out of cloud environments.
-
Migrate data and software across new architecture and/or platforms by defining requirements, designing architectural solutions, porting, modifying, testing, and validating.
-
Troubleshoot and resolve data issues related to software, hardware, network, and data, providing the root cause and recommendations for improvement using all tools and resources available.
-
Perform failovers and validation steps following the approved procedures during both routine and emergency situations.
-
Keep all code, software, and configuration settings identical between all applicable HPC and Virtual Machine (VM) servers unless approved by the Government.
-
Develop and maintain graphing trends for all critical components of data transfers to include but not limited to: transfer rates, volumes, transfer queue sizes, and timeliness of data.
-
Maintain up-to-date documentation for all systems, software, partners, and components to include but not limited to: function, versions, troubleshooting tips, standard procedures, and failover instructions.
-
Create and implement real-time monitoring solutions that detect problems/issues/errors before a customer reports them for all applications, software, and data feeds.
Required Skills/Experiences:
-
Demonstrated experience in migrating high-performance computing (HPC) environments to the cloud
-
Bachelor's degree in a relevant field and 6–8 years of professional experience