Job Title: Data Engineer

Requirements:

1. Bachelor's or master's degree in computer science, engineering, or a related field.
2. Proven experience as a data engineer or a similar role, with a strong background in designing and implementing data solutions.

Mandatory:

• Backend Java
• AWS lambdas and Glue
• Apache Spark for data pipelines
• Glue Experience
• Python is nice to have

Responsibilities:

1. Design, develop, and maintain scalable data pipelines and ETL processes to ingest, transform, and store large volumes of structured and unstructured data from various sources.
2. Collaborate with cross-functional teams to gather and understand data requirements, and translate them into technical specifications for implementation.
3. Implement data models, schemas, and structures optimized for performance, reliability, and scalability.
4. Build and maintain data warehouses, data lakes, and other data storage solutions to ensure data availability and accessibility for analytics and reporting purposes.
5. Optimize data processing workflows and queries for improved efficiency and performance.
6. Monitor and troubleshoot data pipelines and systems to ensure data integrity, reliability, and availability.
7. Work with data governance and security teams to implement data privacy and security measures, ensuring compliance with regulatory requirements.
8. Stay updated with the latest trends and technologies in data engineering, and evaluate and recommend new tools and frameworks to enhance our data infrastructure.
9. Document data architecture, processes, and workflows to facilitate knowledge sharing and collaboration among team members.
10. Provide technical guidance and mentorship to junior team members, and actively contribute to the continuous improvement of our data engineering practices.