What are the responsibilities and job description for the Senior Data Engineer position at Top Stack?
We are seeking a highly skilled Data Engineer to design, develop, and maintain scalable data pipelines and integrations. You will play a critical role in supporting the growth of data usage, volume, and complexity across the organization. This position involves close collaboration with analytics, engineering, and business teams to drive data-informed decisions and optimize operational performance.
Key Responsibilities:
- Develop and maintain scalable and reliable data pipelines and integration tools.
- Design and implement new API integrations to support growing data needs.
- Build analytical tools to derive actionable insights from data, focusing on key business metrics such as operational efficiency and customer acquisition.
- Design and implement data integration frameworks, ensuring high-quality data through integration with monitoring services.
- Integrate and transform data from various systems of record (SORs) into structured, consumable models.
- Analyze data to identify trends, patterns, and insights that support business decision-making.
- Troubleshoot and resolve data-related issues through thorough analysis and collaboration.
- Identify and implement process improvements to enhance scalability, performance, and automation of data systems and pipelines.
- Create solution prototypes and develop data-driven algorithms as needed.
- Establish and enforce data quality processes, ensuring accuracy and availability for stakeholders.
- Collaborate with business and analytics teams to improve data models used in business intelligence platforms.
- Work alongside cross-functional teams including frontend/backend engineers, product managers, and analysts.
Technical Skills & Qualifications:
- Proficiency in SQL and programming languages for handling structured and unstructured data.
- Hands-on experience with Azure services such as ADF, Azure Functions, Azure Service Bus (ASB), Synapse, APIs, and Azure SQL.
- Experience in ETL/ELT processes using Data Lake and Data Warehouse platforms.
- Strong understanding of data modeling, architecture, integration, and database performance tuning.
- Familiarity with Change Data Capture (CDC), data replication, and other modern data integration tools.
- Experience in database administration including performance monitoring, backup, recovery, and security.
- Ability to create and maintain database documentation including data standards and definitions.
- Self-starter with a strong execution and delivery mindset.