What are the responsibilities and job description for the Data Engineer position at Info Gain Consulting?
Who We Are
Info Gain Consulting is a CMMI-DEV L3 appraised, and SBA Certified 8(a) Small business. We deliver program management, financial management, performance improvement, and technology enablement solutions to federal agencies in the US and overseas. Our federal clients value our ability to bring together expertise in people, processes, technology, and data to help them prepare for growth, manage risk, and improve operations.
Position Overview:
We are seeking a skilled Data Engineer to join our team, supporting the Department of Defense (DoD) Office of Inspector General (OIG). The candidate will be responsible for designing, building, and maintaining the infrastructure that supports data storage, processing, and retrieval in the Cloudera Data Platform. They will work with large data sets and develop data pipelines that move data from source systems to data warehouses, data lakes, and other data storage and processing systems. The candidate will also develop and maintain data APIs, ETL processes, and data integration systems.
Responsibilities Include:
Info Gain Consulting is a CMMI-DEV L3 appraised, and SBA Certified 8(a) Small business. We deliver program management, financial management, performance improvement, and technology enablement solutions to federal agencies in the US and overseas. Our federal clients value our ability to bring together expertise in people, processes, technology, and data to help them prepare for growth, manage risk, and improve operations.
Position Overview:
We are seeking a skilled Data Engineer to join our team, supporting the Department of Defense (DoD) Office of Inspector General (OIG). The candidate will be responsible for designing, building, and maintaining the infrastructure that supports data storage, processing, and retrieval in the Cloudera Data Platform. They will work with large data sets and develop data pipelines that move data from source systems to data warehouses, data lakes, and other data storage and processing systems. The candidate will also develop and maintain data APIs, ETL processes, and data integration systems.
Responsibilities Include:
- Data Infrastructure Design: Design and implement robust and scalable data infrastructure solutions within the Cloudera Data Platform.
- Data Pipeline Development: Create and maintain data pipelines to efficiently move data from source systems to data storage and processing systems.
- Data Storage Management: Manage and optimize data storage solutions, including data warehouses and data lakes.
- ETL (Extract, Transform, Load) Processes: Develop and maintain ETL processes to cleanse, transform, and load data into appropriate storage systems.
- Data Integration: Design, develop, and maintain data integration systems to ensure seamless data flow between different components.
- Data API Development: Build and maintain data APIs to facilitate data access for various applications and users.
- Performance Optimization: Continuously monitor and optimize data processes and infrastructure for maximum performance and efficiency.
- Data Security: Implement and maintain data security measures to protect sensitive information.
- Collaboration: Collaborate with cross-functional teams, including data scientists and analysts, to understand data requirements and deliver effective solutions.
- Documentation: Create and maintain clear and comprehensive documentation for data engineering processes and solutions.
- Troubleshooting: Diagnose and resolve data-related issues and provide support as needed.
- Education:
- Bachelor's or higher degree in Computer Science, Information Technology, or a related field.
- Clearance:
- Must be a US Citizen with a current DoD Secret Clearance.
- Certification:
- Must have and maintain a current CompTIA Security CE certification.
- Experience:
- Proven experience in data engineering, including data pipeline development and infrastructure design.
- Proficiency in Cloudera Data Platform or similar big data technologies.
- Strong programming skills in languages such as Python, Java, or Scala.
- Knowledge of database systems and SQL.
- Familiarity with data warehousing concepts and tools.
- Experience with ETL tools and processes.
- Understanding of data security and compliance standards.
- Excellent problem-solving and communication skills.
- Ability to work independently and collaboratively in a remote team environment.
- Certification in relevant data engineering technologies.
- Experience working with the Department of Defense (DoD) or government agencies.
- Knowledge of cloud-based data solutions (e.g., AWS, Azure, GCP).