Data Engineer II

Share job

The Azure Core Organization is responsible for creating the foundation of Microsoft’s Cloud Platform for utility computing. This platform is one of the lowest levels of the services software/hardware stack and includes an efficient, virtualized computational substrate, a fully automated service management system, and a comprehensive set of highly scalable storage services.

The Azure Compute Platform Capacity and Efficiency (AC2E) team is the team in Azure Core organization tasked with managing all aspects of Compute Platform Capacity and Efficiency Management across the fleet. Platform Capacity Management needs to ensure that on the one hand, there is sufficient capacity across all allocation scopes to meet all customer demand, while on the other hand ensuring that capacity is provisioned efficiently thereby avoiding overspending and Cost of Goods Sold/Capital Expenditures (COGS/CAPEX) impact. At the scale of Azure’s business, managing this trade-off across the entire Azure Compute fleet is an enormously complex task, where improvements can make the difference between customer allocation failures on the one hand, and gargantuan savings on the other.

As a Data Engineer in the team, you will work closely with our software engineers, program managers, and data scientists across different teams within Azure Core. You will also collaborate with a variety of internal partner teams across Azure and Microsoft. You will build reliable, secured, highly scalable, performant data pipelines to enhance the Azure Compute allocation, deliver capacity management and efficiency improvements. The value of your work will be reflected in improvements to the Azure platform, Azure service capacity fulfillment rate, customer satisfaction, and various efficiency metrics, including COGS reduction. You will have opportunities for mentorship, accelerate your career growth, and work on truly high-business impact areas.

Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.

Required/Minimum Qualifications:

• Bachelor's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND experience in business analytics, data science, data modeling or data engineering work
  
  • OR equivalent experience.
• Programming Language experience - Python, C# or C++ or equivalent

Other Requirements:

• Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include, but are not limited to the following specialized security screenings:
• Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud Background Check upon hire/transfer and every two years thereafter.

Additional or Preferred Qualifications:

• Master's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND of business analytics, data science, software development, data modeling or data engineering work experience
  • OR equivalent experience

#azurecorejobs

• With guidance, you’ll implement basic code to extract raw data, validate its quality, and ensure the correct data is ingested within your area of work. You’ll also apply basic modification techniques to transform raw data into forms compatible with downstream data sources.
• With guidance from others, you’ll contribute to the development of data tools that can transform, manage, and access data. You’ll also validate code to test the storage and availability of data platforms so that they’re more resilient.
• You will contribute to performance monitoring protocols across a data pipeline, supporting existing visualizations and aggregations to ensure pipeline performance. You’ll assist others in implementing solutions that minimize points of failure and help test improvements across a product feature.
• You will follow data modeling and handling procedures to maintain compliance with all applicable laws and policies across your assigned workstreams. You’ll also learn about permissions and approvals for data access within a data pipeline.
• Data Infrastructure, Implement Continuous Integration/Continuous Delivery (CI/CD) Pipelines, Data Engineering, build metrics, insights generation, data analysis, platform recommendations and analytics with deep domain knowledge understanding.
• Advocate for best practices in data engineering.