What are the responsibilities and job description for the Data Engineer position at TOP Consulting, Inc?
Data Engineer (MISO)
Top Consulting is seeking an experienced Data Engineer with a strong technical background in Microsoft Azure Cloud, Azure Data Factory (ADF), and Azure Synapse Analytics to support data engineering initiatives within the energy industry. This individual will design, develop, and maintain scalable data pipelines and cloud-based data solutions that enable reliable data integration, analytics, and business intelligence. Experience with Microsoft Fabric is preferred.
Responsibilities
o Design, develop, and maintain scalable data pipelines and ETL/ELT processes using Azure Data Factory and Azure Synapse Analytics.
o Build and optimize cloud-based data solutions within Microsoft Azure.
o Integrate data from multiple sources, ensuring data quality, accuracy, consistency, and reliability.
o Develop and maintain data models, data warehouses, and data lake solutions to support reporting and analytics.
o Optimize data workflows, pipeline performance, and processing efficiency.
o Collaborate with data analysts, business stakeholders, and technical teams to translate business requirements into effective data solutions.
o Implement data governance, security, access controls, and data quality standards.
o Troubleshoot data pipeline issues and proactively identify opportunities for performance improvements.
o Support the modernization of data platforms and adoption of emerging Microsoft data technologies.
o Apply industry best practices for cloud architecture, data integration, and data management within the energy sector.
Qualifications:
o Proven experience as a Data Engineer or in a similar data engineering role.
o Strong hands-on technical experience with Microsoft Azure Cloud, Azure Data Factory (ADF), and Azure Synapse Analytics.
o Experience designing, building, and maintaining ETL/ELT pipelines and data integration solutions.
o Proficiency in SQL and experience working with relational databases and data warehousing concepts.
o Experience with data modeling, data transformation, and large-scale data processing.
o Knowledge of cloud-based data architecture, data lakes, and data warehouse solutions.
o Preferred experience with Microsoft Fabric, including Lakehouse, Data Factory, or OneLake capabilities.
o Experience with Azure Databricks, PySpark, or Python for data processing and transformation.
o Preferred experience in energy, utilities, power generation, electricity markets, or related industries.
o Experience implementing data governance, security, and monitoring within cloud environments.
o Knowledge of CI/CD, Git, and automated deployment practices for data engineering solutions.
o Strong analytical, troubleshooting, and problem-solving skills.
o Excellent communication and collaboration skills.