What are the responsibilities and job description for the Data Engineer with Security Clearance position at NS2 Mission?
Are you passionate about building scalable data pipelines and working with complex datasets in cloud environments? NS2 Mission is seeking a skilled Data Engineer to support mission-focused programs in Chantilly, VA. In this role, you will design and maintain data pipelines, collaborate closely with development teams, and help deliver reliable data solutions that enable teams to transform data into actionable insights. Key Responsibilities: * Design, build, and maintain scalable data pipelines that support the ingestion, transformation, and delivery of large datasets
- Collaborate with cross-functional development teams to integrate data solutions into applications and mission systems
- Process and transform data using technologies such as Spark within cloud-based environments
- Develop and optimize SQL queries to support data retrieval, transformation, and analysis
- Work with relational databases including PostgreSQL, Oracle, and MySQL to manage and access structured data
- Create automation and processing scripts using Python, Linux, and shell scripting
- Analyze and work with datasets across multiple formats including CSV, XML, JSON, Avro, and Parquet
- Write and maintain unit tests to validate data processing logic and ensure code quality
- Monitor, troubleshoot, and improve the performance and reliability of data pipelines and workflows Required Qualifications: * 4 years of experience building and managing data pipelines
- 4 years working within highly collaborative development teams
- 2 years of hands-on experience with Python
- Experience working in AWS cloud environments
- Experience processing data using Spark
- Experience working with relational databases such as PostgreSQL, Oracle, or MySQL, including writing SQL queries
- Experience working in Linux environments, including shell scripting
- Experience analyzing data across multiple formats such as CSV, XML, JSON, Avro, and Parquet
- Experience writing and maintaining unit tests Desired Qualifications: * Experience using Apache NiFi, Apache Airflow, or similar orchestration tools to manage data pipelines
- Development experience with Java or Scala
- Experience administering or supporting EMR or Spark clusters
- Background in performance tuning Spark jobs
- Experience working with Hive, Iceberg, or other technologies that provide SQL-based access to large datasets
- Familiarity with implementing cloud-based security solutions
- Experience following configuration management processes for code review and release deployments Technical Environment * Python
- AWS
- Spark
- SQL
- PostgreSQL
- Oracle
- MySQL
- Linux
- Shell Scripting
- Apache NiFi
- Apache Airflow
- Java
- Scala
- Hive
- Iceberg
- EMR
- JSON
- XML
- CSV
- Avro
- Parquet Position Clearance Requirement: TS/SCI with an FSP Please be aware, many of our positions require the ability to obtain or maintain a U.S security clearance which requires U.S citizenship We win with inclusion NS2 Mission's culture of inclusion, focus on health and well-being, and flexible working models help ensure that everyone - regardless of background - feels included and can run at their best. At NS2 Mission, we believe we are made stronger by the unique capabilities and qualities that each person brings to our company, and we invest in our employees to inspire confidence and help everyone realize their full potential. We ultimately believe in unleashing all talent and creating a better world. NS2 Mission is committed to the values of Equal Employment Opportunity and provides accessibility accommodations to applicants with physical and/or mental disabilities. If you are interested in applying for employment with NS2 Mission and are in need of accommodation or special assistance to navigate our website or to complete your application, please send an e-mail with your request to Recruiting Operations Team: . Qualified applicants will receive consideration for employment without regard to their age, race, religion, national origin, ethnicity, age, gender (including pregnancy, childbirth, et al), sexual orientation, gender identity or expression, protected veteran status, or disability, in compliance with applicable federal, state, and local legal requirements.