Design, develop, and maintain scalable data pipelines for ingesting, processing, and transforming data from various sources using Azure Data Factory, Glue, Lambda, and other relevant tools.

Collaborate with cross-functional teams to gather requirements, understand data needs, and translate business objectives into technical solutions.

Implement ETL processes to ensure data quality, consistency, and integrity throughout the data lifecycle.

Utilize Boto3 for programmatically interacting with AWS resources and services as needed.

Manage and optimize data storage solutions including DynamoDB, RDS, and S3 to meet performance, reliability, and cost-efficiency goals.

Leverage EMR for big data processing and analysis tasks, ensuring efficient utilization of resources and timely delivery of insights.

Design and develop data models, queries, and algorithms to support ad-hoc analysis, reporting, and visualization needs.

Create interactive and insightful dashboards and reports using Tableau and/or Power BI to communicate findings and facilitate data-driven decision-making.

Monitor data pipelines, troubleshoot issues, and implement proactive measures to ensure system stability and reliability.

Stay updated with industry trends, best practices, and emerging technologies in data engineering, analytics, and visualization.