Job Description
Responsibilities:
The Data Engineer will support data collection, ingestion, validation, and loading of optimized data in the appropriate data stores. They work on a team made up of analyst(s), developer(s), data scientist(s), and product leads, and everyone on the team collaborates in support of a specific mission. Working directly with the analyst(s) and the product lead, the data engineer identifies and implements solutions for the data requirements, including building pipelines to collect data from disparate, external sources and implementing rules to validate that expected data is received, cleansed, transformed, massaged and in an optimized output format for the data store. The Data Engineer performs validation and analytics corresponding with client requirements and evolves solutions through automation, optimizing performance with minimal human involvement. As pipelines are executed, the data engineer monitors their status, and performance, and troubleshoots issues while working on improvements to ensure the solution is the very best version to address the customer need. Required Skills:
1. Clearance: Secret Clearance
2. 6 years of experience with SQL
3. 6 years of experience developing data pipelines using modern Big Data ETL technologies like NiFi or StreamSets
4. 6 years of experience with a modern programming language such as Python or Java
5. 6 years of experience working in a big data and cloud environment
6. Documented experience with AWS, EC2, S3, and/or RDS
Preferred Skills:
1. 3 years of experience working in an agile development environment
2. Ability to quickly learn technical concepts and communicate with multiple functional groups.
3. Ability to display a positive, can-do attitude to solve the challenges of tomorrow.
4. Possession of excellent verbal and written communication skills
5. Preferred experience at the respective command with an understanding of analytical and data paint points and challenges across the J-Codes.