Senior applied scientist – outlook copilot
overview
we are seeking a senior applied scientist to join the outlook copilot initiative, focusing on areas such as large language models (llms), prompt engineering, fine-tuning, evaluation, relevance, and responsible ai (rai). This role involves developing end-to-end infrastructure and measurement frameworks, fostering cross-functional collaboration, and leveraging data science and ai expertise to guide strategic decisions. The successful candidate will work with multiple organizations to drive evaluation and optimization of llm systems and related components.
responsibilities
* strategic technical leadership: develop and execute a comprehensive strategy for llm evaluation, covering quality, cost, model performance, utility (user experience and prompt effectiveness), and responsible ai considerations. Drive cost-effective solutions such as small language models (slms) or fine-tuned models.
* data science expertise: apply data science skills to design experiments, analyze data, define okrs, and establish measurement frameworks. Derive actionable insights to improve llm systems and influence product direction and user experience based on evaluation outcomes.
* model and prompt evaluation: lead efforts to assess and enhance the performance and effectiveness of language models and prompts. Drive iterative improvements, including the creation of synthetic and curated datasets.
* program management: oversee large-scale, cross-functional evaluation programs, ensuring alignment with goals and timelines. Develop and maintain robust measurement frameworks to track llm performance and user impact. Partner with engineering teams to build automated evaluation pipelines.
* user experience enhancement: collaborate with ux teams to evaluate and optimize user interactions with ai systems, improving satisfaction and usability.
* responsible ai (rai): implement responsible ai and dsb principles to ensure ethical, unbiased practices in model development and deployment.
* research contributions: lead deep research initiatives in llm evaluation and user experience optimization. Contribute to the scientific community and strengthen alignment with user mental models and llm-powered experiences.
* cross-functional collaboration: work with engineering, research, and product teams to integrate evaluation processes into the development lifecycle.
qualifications
required qualifications:
* bachelor's degree in statistics, econometrics, computer science, electrical or computer engineering, or related field and 4+ years of related experience (e.g., statistics, predictive analytics, research) or master’s degree in the same fields and 3+ years of related experience or doctorate in the same fields and 1+ year of related experience or equivalent experience.
* extensive experience in data science, machine learning, experimentation, and ai, with a strong track record of delivering impactful results.
* expertise in llm finetuning, reinforcement learning, evaluation techniques, implementing rag techniques, agentic workflows and industry best practices.
* proven expertise in program management and leading cross-functional teams.
* proficient development experience in python.
* proficient understanding of responsible ai principles.
preferred qualifications:
* ability to work in a fast-paced and dynamic environment.
* excellent analytical and problem-solving skills.
* exceptional communication and presentation abilities.
other requirements
ability to meet microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the microsoft cloud background check, which is required upon hire/transfer and every two years thereafter.
microsoft is an equal opportunity employer. Consistent with applicable law, all qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.
seniority level
* not applicable
employment type
* full-time
job function
* research, analyst, and information technology
industries
* software development
referrals increase your chances of interviewing at microsoft by 2x
senior software engineer (outlook services) • data scientist / applied scientist (mid and senior levels)
#j-18808-ljbffr