Senior Data Developer

Senior Data Developer (Data Engineer)

Job Title: Senior Data Developer

Department: Risk

Reports to: Director, Data Analytics & Business Intelligence

FLSA Status: Exempt (Salaried)

Summary

The Senior Data Developer will play a key role in designing, developing, and maintaining data integration solutions using SQL Server, Microsoft SQL Server Integration Services (SSIS), Azure Data Factory, and other tools to extract, transform, and load data from various sources into integrated enterprise solutions, focused on building efficient and reliable ETL pipelines for business intelligence purposes.  This individual will lead the design of the logical data model, implementation of the physical database structure, as well as the construction and implementation of operational data stores, data marts, and data models.  This individual will seek to gain a deep understanding of the organization’s data to skillfully carry out the company's integrated data design projects.

Essential Duties and Responsibilities

SQL development: Writing complex SQL queries, stored procedures, and triggers to manage data within the database.

Database design and architecture: Creating data models, defining schemas, tables, relationships, and constraints to optimize data storage and retrieval. Explain design decisions to others.

Data Analysis and Requirements Gathering:  Collaborate with business analysts and stakeholders to understand data requirements, identify data sources, and translate them into effective ETL processes and data stores.

ETL Design and Development:  Design and implement complex ETL solutions utilizing data flow tasks, control flow tasks, data transformations (like lookups, derived columns, aggregations), and data cleansing techniques to ensure data quality.

Data Source Integration:   Extract data from diverse sources including relational databases, flat files, APIs, web services, and other applications using appropriate connectors.

Data Transformation:   Perform data cleansing, manipulation, and aggregation operations to prepare data for loading into target systems, including data validation, deduplication, and data type conversion.

Performance Optimization:   Monitor and optimize ETL processes to improve performance and scalability, identifying bottlenecks and implementing efficient data loading strategies.

Deployment and Scheduling:  Deploy solutions to production environments and schedule automated data refreshes based on business needs.

Documentation and Maintenance:  Maintain detailed documentation of ETL processes, data mappings, and configurations, and troubleshoot any issues that arise in production environments.

Data Accuracy:  Ensure all processes and data outputs are accurate, through source system comparisons and verification by business stakeholders.   Maintain appropriate dev, qa, change control, and production processes and environments to allow for complete testing and rollback capabilities.

Database Security: Implementing security measures to protect sensitive data, including user access controls and encryption. Manage database access and permissions.

Backup and Recovery: Creating and managing database backups to ensure data availability in case of system failures.

Data Models: Create views, stored procedures, multidimensional database (MDB), and PowerBI semantic models to be used by reporting and analytics teams.

Education and Experience

Bachelor’s degree in Computer Science, Information Technology, and/or Engineering is preferred.

5 years as an ETL Developer, Data Engineer, Database Architect, or similar role

Experience with SQL, SSMS, SSIS, SSRS, Power BI, dimensional modeling, OLAP, MDX, and Azure Data Factory.

Skills

Advanced SQL Skills: Strong understanding of SQL syntax, SQL query writing, stored procedures, functions, query optimization, and data manipulation techniques to extract and manipulate data from various sources.

Proficient in SQL Server Integration Services (SSIS): Comprehensive knowledge of SSIS components, tasks, and features including data flow tasks, control flow tasks, data transformations, and package configuration.

Proficient in Azure Data Factory.

Database management systems (DBMS): Expertise in specific database platforms like Oracle, SQL Server, etc.

Data Warehousing Concepts: Knowledge of data warehouse design principles, dimensional modeling (star schema, snowflake schema) and data mart concepts.

Data Quality Assurance: Ability to implement data quality checks and validation rules to ensure data integrity throughout the ETL process.

Problem-Solving and Troubleshooting: Effective analytical skills to diagnose and resolve issues within ETL pipelines.

Database design principles: Ability to design logical and physical data models. Knowledge of normalization, data integrity, and transaction management.  Knowledge of techniques for enhancing database scalability and efficiency.

Programming languages: Familiarity with programming languages used to interact with databases (e.g., Java, Python, .NET)

Cloud database technologies: Understanding of cloud-based database services like AWS RDS, Azure SQL Database, etc.

Analytical skills: Ability to analyze data and identify trends to improve database design and performance.

Communication skills: Effective communication with both technical and non-technical stakeholders to gather requirements and explain complex database concepts.  Ability to present designs both visually and discursively.