Overview
outlook is one of the most widely used communication and productivity tools in the world, and copilot is transforming how millions of users interact with it through the power of ai. We are seeking a senior applied scientist to join a team of exceptional experts at the forefront of this transformation. In this role, you will advance outlook’s copilot initiatives in areas such as large language models (llms), prompt engineering, fine-tuning, evaluation, relevance, and responsible ai (rai). You will develop end-to-end infrastructure and measurement frameworks, foster cross-functional collaboration, and leverage data science and ai expertise to guide strategic decisions. You will work closely with multiple organizations and stakeholders to drive the evaluation and optimization of our llm systems and related components.
microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. We build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.
responsibilities
* strategic technical leadership: develop and execute a comprehensive strategy for llm evaluation, covering quality, cost, model performance, utility (user experience and prompt effectiveness), and responsible ai considerations. Ensure alignment with company-wide initiatives and leverage emerging research. Drive cost-effective solutions such as small language models (slms) or fine-tuned models.
* data science expertise: apply strong data science skills to design experiments, analyze data, define okrs, and establish measurement frameworks. Derive actionable insights to improve llm systems and influence product direction and user experience based on evaluation outcomes.
* model and prompt evaluation: lead efforts to assess and enhance the performance and effectiveness of language models and prompts. Drive iterative improvements, including the creation of synthetic and curated datasets.
* program management: oversee large-scale, cross-functional evaluation programs, ensuring alignment with organizational goals and timelines. Develop and maintain robust measurement frameworks to track and report llm performance and user impact. Partner with engineering teams to build automated evaluation pipelines integrated into product workflows.
* user experience enhancement: collaborate with ux teams to evaluate and optimize user interactions with ai systems, improving satisfaction and usability.
* responsible ai (rai): implement responsible ai and dsb principles to ensure ethical, unbiased practices in model development and deployment.
* research contributions: form partnerships and lead deep research initiatives in llm evaluation and user experience optimization. Contribute to the scientific community and strengthen the team’s understanding of user mental models and alignment with llm-powered experiences.
* cross-functional collaboration: work closely with engineering, research, and product teams to seamlessly integrate evaluation processes into the development lifecycle.
qualifications
required qualifications:
* bachelor's degree in statistics, econometrics, computer science, electrical or computer engineering, or related field and 4+ years of related experience (e.g., statistics, predictive analytics, research) or master’s degree in statistics, econometrics, computer science, electrical or computer engineering, or related field and 3+ years of related experience (e.g., statistics, predictive analytics, research) or doctorate in statistics, econometrics, computer science, electrical or computer engineering, or related field and 1+ year of related experience (e.g., statistics, predictive analytics, research) or equivalent experience.
* extensive experience in data science, machine learning, experimentation, and ai, with a strong track record of delivering impactful results.
* expertise in llms with finetuning, reinforcement learning, evaluation techniques, implementing rag techniques, agentic workflows and industry best practices.
* proven expertise in program management and leading cross-functional teams.
* proficient development experience in python.
* proficient understanding of responsible ai principles.
preferred qualifications:
* ability to work in a fast-paced and dynamic environment.
* excellent analytical and problem-solving skills.
* exceptional communication and presentation abilities.
other requirements:
* ability to meet microsoft, customer and/or government security screening requirements. These requirements include but are not limited to the following specialized security screenings: microsoft cloud background check. This position will be required to pass the microsoft cloud background check upon hire/transfer and every two years thereafter.
microsoft is an equal opportunity employer. Consistent with applicable law, all qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.
#j-18808-ljbffr