What are the responsibilities and job description for the Azure DevOps / Data Support Engineer position at Nsight?
Job Details
Overview
We are seeking a skilled Azure DevOps / Data Support Engineer to manage and optimize data pipelines, streaming analytics, and cloud-based integration services. The ideal candidate will have hands-on experience with Microsoft Azure data services, strong troubleshooting skills, and a proactive approach to monitoring and data operations.
Key Responsibilities
- Develop, monitor, and maintain data ingestion, transformation, and integration pipelines within Azure Data Factory (ADF) and Azure Data Explorer (ADX).
- Create and optimize KQL queries for performance tuning and analytics.
- Manage Azure Blob Storage operations, including automation using AzCopy for large-scale data movement.
- Build and maintain real-time data streaming and processing solutions using Azure Stream Analytics, Azure Functions, and Kafka.
- Configure and support Logstash pipelines for data ingestion and transformation from various sources.
- Collaborate with DevOps and Data Engineering teams to ensure reliable CI/CD processes for data workloads.
- Troubleshoot production issues, implement monitoring, and support incident resolution.
- Maintain system documentation, standard operating procedures, and knowledge base articles.
Must Have Skills
- Proficiency in Microsoft Azure ecosystem especially:
- Azure Data Factory (ADF)
- Azure Data Explorer (ADX) and Kusto Query Language (KQL)
- Azure Blob Storage and AzCopy for data management
- Azure Stream Analytics, Azure Functions
- Experience with Kafka (producers, consumers, topics, partitions, etc.)
- Hands-on experience with Logstash configuration and troubleshooting.
Nice to Have
- Familiarity with Google Cloud Platform (Google Cloud Platform) and BigQuery for cross-cloud data operations.
- Exposure to Datadog for performance monitoring and observability.
- Experience with CI/CD pipelines and infrastructure as code (IaC) (e.g., Terraform, ARM templates, or Bicep).
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
