NexSolv is looking for Sr. Data Engineer for our direct state client. Onsite interview in Linthicum, MD required. Locals preferred.

Duties:

• Designing, implementing, and maintaining batch and streaming data pipelines between various SQL sources, including Aurora and SQL Server, and a target data lake in S3, as well as enterprise data stored in Redshift/Snowflake.
• Expertise in AWS cloud services, such as Kinesis, S3, Lake Formation, Glue, and Step Functions, to build scalable, reliable, and high-performance data pipelines that enable seamless data integration and empower data-driven insights.
• Data Pipeline Design: Design end-to-end data pipelines that efficiently extract, transform, and load data from SQL sources (Aurora, SQL Server) to the target data lake in S3 and the enterprise data in Redshift/Snowflake.
• Batch and Streaming Integration: Implement both batch and real-time streaming data integration solutions using AWS Kinesis and other relevant technologies.
• Data Transformation: Develop data transformation processes using AWS Glue or other ETL tools to harmonize, cleanse, and enrich data for analytical use.
• Data Lake Management: Oversee the setup and configuration of the data lake in S3, applying AWS Lake Formation best practices for data organization, cataloging, and access control.
• Data Governance: Ensure adherence to data governance and security standards across the data pipelines, guaranteeing data privacy and compliance.
• Performance Optimization: Continuously monitor and optimize the performance of the data pipelines, addressing bottlenecks and ensuring efficient data processing and delivery.
• Error Handling and Monitoring: Implement error handling mechanisms and robust data monitoring to identify and resolve data pipeline issues proactively.
• Data Cataloging and Lineage: Establish and maintain data cataloging and lineage information using AWS Glue Data Catalog to enable data discoverability and traceability.
• Documentation: Create comprehensive technical documentation, including design specifications, data flow diagrams, and operational guides.
• Collaboration: Collaborate with data analysts, data scientists, and other stakeholders to understand data requirements and deliver reliable data solutions.
• Data Governance: Ensure data governance principles are implemented throughout the data pipelines to maintain data quality and integrity.

Education Qualification:

• Bachelor's Degree from an accredited college or university with a major in Computer Science, Information Systems, Engineering, Business, or other related scientific or technical discipline.

General Experience:

• At least six (6) years of experience working on AWS cloud-based batch and streaming data pipelines.
• Strong proficiency in AWS cloud services, including Kinesis, S3, Lake Formation, Glue, and Step Functions.
• In-depth knowledge of SQL databases, such as Aurora and SQL Server, and data lakes in S3, as well as enterprise data in Redshift/Snowflake.
• Hands-on experience with ETL tools, data transformation, and data integration techniques.
• Familiarity with data governance, data privacy, and security best practices in AWS environments.
• Strong problem-solving skills and the ability to troubleshoot complex data pipeline issues.