What are the responsibilities and job description for the Lead Data Engineer / Data Architect position at Luxoft?
Project Description:
The primary goal of the project is the modernization, maintenance and development of an eCommerce platform for a big US-based retail company, serving millions of omnichannel customers each week.
Solutions are delivered by several Product Teams focused on different domains - Customer, Loyalty, Search and Browse, Data Integration, Cart
.
Current overriding priorities are new brands onboarding, re-architecture, database migrations, migration of microservices to a unified cloud-native solution without any disruption to busines
s.
Responsibiliti
es:We are looking for Data Architect who will be responsible for designing a solution for a big retail company. The main focus is to support processing of big data volumes and integrate solution to current architectu
re.
Mandatory Skills Descript
- ion:Readiness to work until 8.00 pm CET (no need to do overti
- mes)Overall years of experience required 8 (at least 1 year in an Architect posit
- ion)Strong, recent hands-on expertise with Azure Data Factory and Synapse is a must (3 yea
- rs).Strong expertise in designing and implementing data models, including conceptual, logical, and physical data models, to support efficient data storage and retrie
- val.Strong knowledge of Microsoft Azure, including Azure Data Lake Storage, Azure Synapse Analytics, Azure Data Factory, and Azure Databricks, pySpark for building scalable and reliable data soluti
- ons.Extensive experience with building robust and scalable ETL/ELT pipelines to extract, transform, and load data from various sources into data lakes or data warehou
- ses.Ability to integrate data from disparate sources, including databases, APIs, and external data providers, using appropriate techniques such as API integration or message queu
- ing.Proficiency in designing and implementing data warehousing solutions (dimensional modeling, star schemas, Data Mesh, Data/Delta Lakehouse, Data Va
- ult)Proficiency in SQL to perform complex queries, data transformations, and performance tuning on cloud-based data stora
- ges.Experience integrating metadata and governance processes into cloud-based data platf
- ormsCertification in Azure, Databricks, or other relevant technologies is an added advan
- tage•Experience with cloud-based analytical databa
- ses.Experience with Azure MI, Azure Database for Postgres, Azure Cosmos DB, Azure Analysis Services, and Infor
- mix.Experience with Python and Python-based ETL to
- ols.Experience with shell scripting in Bash, Unix or windows shell is prefera
- ble.Demonstrated ability to lead cross-functional engineering teams, define technical strategy and architecture, drive delivery of complex data platforms, mentor engineers, and effectively communicate with stakeholders at all organizational lev
els.
Nice-to-Have Skills Descrip
- tion:Experience with Elastics
- earchFamiliarity with containerization and orchestration technologies (Docker, Kuberne
- tes).Troubleshooting and Performance Tuning: Ability to identify and resolve performance bottlenecks in data processing workflows and optimize data pipelines for efficient data ingestion and anal
- ysis.Collaboration and Communication: Strong interpersonal skills to collaborate effectively with stakeholders, data engineers, data scientists, and other cross-functional t
- eams.Ability to plan, estimate and track progress of implementing fea
- turesComputer Science and data science academic and education creden