Data Engineer - CUI Data Integration and Mining

About the Role:

We are seeking highly motivated Data Engineers to join our team and play a vital role in integrating a massive repository (petabyte ) of historical Controlled Unclassified Information (CUI) data. This data will reside within a secure Azure Government environment, and the successful candidate will be able to pass a Tier 2 / MBI background check process. Candidates with an active security clearance will be given priority consideration.

Leveraging your expertise in data engineering, cloud technologies (Azure), and big data processing, you will design, develop, and implement efficient data pipelines to extract, transform, and load (ETL) this CUI data for further analysis.

Responsibilities:

• Develop and implement scalable and secure data pipelines using Azure Data Factory or similar tools for efficient extraction, transformation, and loading (ETL) of CUI data while adhering to all security protocols.
• Utilize big data processing tools and techniques to explore and analyze massive datasets while maintaining data integrity and adhering to CUI regulations.
• Work closely with system administrators to ensure secure access to the CUI data repository within the Azure Government environment.
• Document data pipelines, workflows, and technical specifications to facilitate knowledge sharing and maintainability.
• Collaborate with data team members and security specialists to understand CUI data requirements and design secure data access and processing strategies.
• Stay up-to-date with emerging technologies and best practices in data engineering, cloud platforms, and CUI data security protocols.

Qualifications:

• Minimum 2 years of experience as a Data Engineer or similar role (there are multiple positions, and candidates with more experience are strongly encouraged to apply)
• Candidates must be able to pass a Tier 2/MBI background investigation
• Experience with Databricks
• Proficiency in Python for data manipulation and analysis.
• Experience with cloud platforms like Microsoft Azure, particularly Azure Data Factory and security features within Azure Government.
• Proven ability to rapidly and accurately integrate new or foreign data sources with established and evolving data solutions.
• In-depth understanding of data security best practices and regulations, especially those pertaining to CUI data handling.
• Experience with big data processing frameworks and tools (e.g., Apache Spark, Hadoop).

Preferred Skills:

• Experience working with sensitive data in a government or secure environment.
• Familiarity with data mining techniques and machine learning algorithms.
• Excellent communication and collaboration skills to work effectively with cross-functional teams (data scientists, security specialists, system administrators).
• Candidates with an active security clearance will be given priority review

Benefits:

We offer a competitive compensation and benefits package, along with the opportunity to work on challenging and impactful projects that utilize cutting-edge technologies while safeguarding sensitive information.

Join our team and be a part of unlocking the valuable insights within this vast CUI data repository!

Please note: Due to the nature of this role and the sensitivity of the data, a Tier 2/MBI or higher adjudicated security clearance is mandatory.

This is a remote position. Data and systems may only be accessed using client provided systems.

If you are interested or know of someone that is interested, please contact us immediately. We offer a "paid for referral program" that pays you if you refer a candidate that is hired.