Senior Data Engineer

Microsoft runs on trust. Earning and keeping that trust has never been more important.
In Customer Security and Trust (CST), program managers, engineers, analysts, investigators, data scientists, attorneys, and business professionals are responsible for some of the most exciting projects at Microsoft focused on protecting our customers.
- We actively partner with law enforcement and security organizations around the globe to disrupt bad actors who are using malware, phishing scams, IP abuse, tech support fraud, and other means to target our customers.
- We ensure that governments who are requesting data from Microsoft in support of their criminal investigations through legal orders follow due process, striking a balance between fighting cybercrime and protecting customer data.
- We provide solutions to demonstrate that Microsoft is in compliance with cybersecurity practices in support of government and industry customers.
Our CST Engineering team is growing. We are looking for a Senior Data Engineer to join our team and help deliver on our vision of creating a safer digital experience for every person and organization on the planet, which in turn, promotes trust in Microsoft.
As part of the CST Engineering team, you will be hands-on building products from inception to launch and everything in-between.
The successful candidate will have a strong sense of teamwork, personal integrity, accountability, and the ability to understand business functions and requirements, translating to innovative working applications while navigating competing priority tradeoffs
Responsibilities
- Leads collaborations with appropriate stakeholders across product groups and uses understanding of tradeoffs and project costs to recommend appropriate data requirements.
- Applies in-depth knowledge of the business to design a data model that is appropriate for the project and translates business requirements into design specification documents to model the flow and storage of data across multiple data pipelines.
- Identifies multiple, complex data sources and builds advanced code to extract raw data from identified upstream sources using query languages, tools, or machine learning algorithms, while assuring accuracy, validity, and reliability of the data across the pipeline.
- Uses and contributes to the development of advanced and reusable modification techniques and operations.
- Collaborates with others to identify opportunities to optimize data tools used to transform, manage, and access data across teams. Ensures the effectiveness and placement of performance monitoring protocols across multiple data pipelines.
- Leads root cause analysis in response to detected problems/anomalies to identify the reason for alerts and implement advanced solutions that prevent recurring points of failure.
- Anticipate the need for data governance and works with designated committees to ensure that data modeling and data handling procedures are compliant with applicable laws and policies across responsibilities in the data pipeline.
- Uses technical expertise of existing tools' strengths and limitations to improve data tools used to transform, manage, and access data across the organization
- Leads the negotiation of complex agreements with partners and system owners to align on project delivery, data ownership between both parties, and the shape and cadence of data extraction for a complex product
- Embody our culture (https://careers.microsoft.com/us/en/culture) and a demonstrated commitment to the company's core values (https://www.microsoft.com/en-us/about/corporate-values) , including a commitment to high ethical standards, unquestioned integrity, and a commitment to advancing the company's diversity & inclusion commitments.
Qualifications
Required Qualifications:
- Bachelor's Degree in Computer Science or related field AND 4 years of experience in business analytics, data science, software development, data modeling and/or data engineering work OR Master's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 3 years of experience in business analytics, data science, software development, data modeling and/or data engineering work OR equivalent experience.
- Experience with a combination of tools/tech including but not limited to: C#, SQL Server, and Python.
- Experience working across the data stack on a variety of complex projects, while providing technical and thought leadership.
- 3 years of experience in the areas of Graph databases, and Big Data or Hyper Scale processing offering in Azure.
- 4 years of experience in the BI/Analytics space using a combination of tools/tech including but not limited to NoSQL, PowerBI, Kusto, Azure Synapse Analytics, Data Explorer, Cosmos, App Insights, and Snowflake (or similar).
- Experience developing best practices on designing components of the data model for a functional area of a project.
- The successful candidate must be a U.S. Citizen.
Preferred Qualifications:
- Technical expertise of existing tools' strengths and limitations to improve data tools used to transform, manage, and access data across the organization.
- Excellent attention to detail with desire to learn and master new technologies.
- Ability to organize, schedule work and communicate effectively in a fast-paced environment, while maintaining confidentiality.
- Collaborate effectively with product management, program management, engineers, and stakeholders.
- Excellent analytical and critical thinking skills
- Ability to Interprets laws and works with engineers to understand and develop data modeling and data management, access, visualization procedures that ensure compliance with applicable laws, and policies across the whole data pipeline
- Experience with microservices architectures and container technologies, such as docker and Kubernetes.
- Expertise in the areas of CI/CD, Continuous testing, and site reliability.
- Working experience in AI/ML Ops.
- Ability to influence without authority and thrive in an ambiguous environment.
#CELA
Citizenship Verification: This position requires verification of US Citizenship to meet federal government security requirements.
Data Engineering IC4 - The typical base pay range for this role across the U.S. is USD $112,000 - $218,400 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $145,800 - $238,600 per year.
Microsoft has different base pay ranges for different work locations within the United States, which allows us to pay employees competitively and consistently in different geographic markets (see below). The range above reflects the potential base pay across the U.S. for this role (except as noted below); the applicable base pay range will depend on what ultimately is determined to be the candidate's primary work location. Individual base pay depends on various factors, in addition to primary work location, such as complexity and responsibility of role, job duties/requirements, and relevant experience and skills. Base pay ranges are reviewed and typically updated each year. Offers are made within the base pay range applicable at the time.
At Microsoft certain roles are eligible for additional rewards, including merit increases, annual bonus and stock. These awards are allocated based on individual performance. In addition, certain roles also have the opportunity to earn sales incentives based on revenue or utilization, depending on the terms of the plan and the employee's role. Benefits/perks listed here may vary depending on the nature of employment with Microsoft and the country work location. U.S.-based employees have access to medical, dental, and vision insurance, a 401(k) plan and company match, short-term and long-term disability coverage, basic life insurance, and wellbeing benefits, among others. U.S.-based employees also receive up to 10 paid holidays, accrue up to 15 paid vacation days, and receive up to 80 hours paid sick time (front-loaded) per calendar year.
Our Commitment to Pay Equity
We are committed to the principle of pay equity - paying employees equitably for substantially similar work. To learn more about pay equity and our other commitments to increase representation and strengthen our culture of inclusion, check out our annual Diversity & Inclusion Report. ( https://www.microsoft.com/en-us/diversity/inside-microsoft/annual-report )
Understanding Roles at Microsoft
The top of this page displays the role for which the base pay ranges apply - Data Engineering IC4.
The way we define roles includes two things: discipline (the type of work) and career stage (scope and complexity). The career stage has two parts - the first identifies whether the role is a manager (M), an individual contributor (IC), an admin-technician-retail (ATR) job, or an intern. The second part identifies the relative seniority of the role - a higher number (or later letter alphabetically in the case of ATR) indicates greater scope and complexity.
Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, color, family or medical care leave, gender identity or expression, genetic information, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran status, race, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable laws, regulations and ordinances. We also consider qualified applicants regardless of criminal histories, consistent with legal requirements. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request via the Accommodation request form (https://careers.microsoft.com/us/en/accommodationrequest) .
Benefits/perks listed below may vary depending on the nature of your employment with Microsoft and the country where you work.