What are the responsibilities and job description for the Principal Data Engineer position at CBSLLC?
The Data Engineer role will work closely with an agile development team to analyze and optimize ETL processes, tables, views, materialized views, and stored procedures to create and maintain high-performance logical data warehouse solutions. This role will be working with Microsoft-focused technologies including Microsoft SQLServer, Azure Data Services including Azure Fabric, Azure Data Lake, AzureDB, Power Platform Integrations. The Data Engineer will focus on optimizing existing systems as well as developing designs and strategies for implementing new systems. This role will be responsible for requirements gathering, analysis, data categorization, interpreting requirements, and developing development tasks for sprint execution.
Essential Functions:
• Design, implement, test, and deploy data processing infrastructure on premise and in cloud
• Contribute to architecture of highly scalable and reliable data engineering solutions for moving large data efficiently across systems
• Perform work in an Agile team setting
• Break down, estimate, and provide just-in-time design for small increments of work
• Work in a Microsoft Azure data infrastructure environment
• Develop in-depth data pipeline using industry-standard data integration tools
• Full development life cycle management, including gathering, analysis, architecture, design, implementation, testing, deployment, and technical support
• Write test cases and test scripts for data quality assurance
• Responsible for creating stored procedures and functions
• Develop dimensional data model with the industry-standard tool
• Interpret reporting requirements into actionable development tasks
• Analyze and optimize SQL based stored procedures and jobs
• Analyze table indices for performance
• Design and implement Materialized Views and Views
• Develop high-performance programs and procedures for ETL Processes
• Analyze, categorize and document data sources and elements
• Write and optimize queries and provide guidance to other developers accessing data
• Agile development experience required, must be comfortable working with a distributed team