What are the responsibilities and job description for the Data Engineer position at Microgreen Technologies LLC?
Job Details
Role : Azure Data Engineer
Mode of Work: Contract on W2 only (No C2C)
Must Skills: Azure Data Factory, Azure Synapse, Azure SQL, Azure Data Lake, and Azure App Service
Job description:
- Experience with complex data modelling, ETL design, and using large databases in a business environment
- Experience with building data pipelines and applications to stream and process datasets at low latencies
- Fluent with Big Data technologies like Spark, Kafka and Hive
- Expert level understanding on Azure Data Factory, Azure Synapse, Azure SQL, Azure Data Lake, and Azure App Service is required
- Designing and building of data pipelines using API ingestion and Streaming ingestion methods
- Knowledge of Dev-Ops processes (including CI/CD) and Infrastructure as code is essential
- Experience in developing NO SQL solutions using Azure Cosmos DB is essential
- Working knowledge of Python is desirable
- Expert level skills writing and optimizing complex SQL
- Designing and implementing scalable and secure data processing pipelines using Azure Data Factory, Azure Databricks, and other Azure services
- Managing and optimizing data storage using Azure Data Lake Storage, Azure SQL Data Warehouse, and Azure Cosmos DB
- Implementing data security measures, including encryption, access controls, and auditing, to protect sensitive information
- Automating data pipelines and workflows to streamline data ingestion, processing, and distribution tasks
- Utilizing Azure's analytics services, such as Azure Synapse Analytics, to provide insights and support data-driven decision-making.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.