What are the responsibilities and job description for the Data Engineer position at Delta System & Software, Inc.?
Data Engineer – Hadoop | Washington, DC | Active Top Secret Clearance | 5 Days Onsite
Job Description:
- Design, develop, implement, and maintain enterprise-scale Hadoop-based Big Data solutions using HDFS, MapReduce, YARN, and Hadoop ecosystem technologies to process large volumes of structured and unstructured data.
- Develop and optimize distributed data processing workflows using Apache Hive, Pig, Sqoop, Flume, and Oozie for efficient data ingestion, transformation, storage, and processing.
- Build, manage, and optimize complex ETL data pipelines within Hadoop clusters, ensuring data quality, performance tuning, fault tolerance, scalability, and high availability.
- Perform Hadoop cluster administration, configuration, monitoring, troubleshooting, and performance optimization while managing distributed storage and computing environments.
- Collaborate with data architects, analysts, developers, and federal stakeholders to deliver secure Big Data solutions while ensuring compliance with government security policies and data governance standards. Must possess an Active Top Secret Security Clearance and be available to work onsite in Washington, DC, five days per week.