What are the responsibilities and job description for the Data engineer Senior DataBricks/SQL position at Luxoft?
Project Description:
- • Work closely with source data application teams and product owners to design, implement and support analytics solutions that provide insights to make better decisions
- • Implement data migration and data engineering solutions using Azure products and services: (Azure Data Lake Storage, Azure Data Factory, Azure Functions, Event Hub, Azure Stream Analytics, Azure Databricks, etc.) and traditional data warehouse tools
- • Perform multiple aspects involved in the development lifecycle - design, cloud engineering (Infrastructure, network, security, and administration), ingestion, preparation, data modeling, testing, CICD pipelines, performance tuning, deployments, consumption, BI, alerting, prod support
- • Provide technical leadership and collaborate within a team environment as well as work independently
- • Be a part of a DevOps team that completely owns and supports their product
- • Implement batch and streaming data pipelines using cloud technologies
- • Understands and follow development of coding standards, best practices and privacy and security guidelines
Responsibilities:
- • Work closely with source data application teams and product owners to design, implement and support analytics solutions that provide insights to make better decisions
- • Implement data migration and data engineering solutions using Azure products and services: (Azure Data Lake Storage, Azure Data Factory, Azure Functions, Event Hub, Azure Stream Analytics, Azure Databricks, etc.) and traditional data warehouse tools
- • Perform multiple aspects involved in the development lifecycle - design, cloud engineering (Infrastructure, network, security, and administration), ingestion, preparation, data modeling, testing, CICD pipelines, performance tuning, deployments, consumption, BI, alerting, prod support
- • Provide technical leadership and collaborate within a team environment as well as work independently
- • Be a part of a DevOps team that completely owns and supports their product
- • Implement batch and streaming data pipelines using cloud technologies
- • Understands and follow development of coding standards, best practices and privacy and security guidelines
- Mandatory Skills Description:
- Top required skills:
- Databricks, Python, SQL Cloud engineering
- • Bachelor's degree in Computer Science, Engineering, Information Systems, or related field (or equivalent experience).
- • 3 to 5 years of experience in software development (Agile, DevOps), designing, developing, and implementing large-scale applications or data engineering solutions.
- • 3 to 5 years of data analytics experience using SQL and working with data platforms like Teradata, Cassandra, MongoDB, Oracle, SQL Server, Snowflake, ADLS, and Azure Data Explorer.
- • 3 to 5 years of cloud development and data lake experience, primarily in Microsoft Azure (EventHub, Data Factory, Databricks, Blob Storage, Power BI, Azure Functions, etc.).