Applied Scientist I

Are you passionate about advancing the science of evaluating large language models and intelligent agents? Join **Thomson Reuters Labs**, where we experiment, build, and deliver cutting-edge AI systems that empower professionals worldwide.

Our flagship AI assistant, **CoCounsel**, helps legal, tax, and business professionals work smarter. We’re expanding our **LLM Evaluation team**, focused on developing automated, scalable, and trustworthy evaluation frameworks that measure model reasoning, reliability, and alignment.

**What We Do**:
At **Thomson Reuters Labs**, we blend **applied research** with real-world impact. Our scientists work on projects spanning **LLM reasoning, benchmarking, grounding, and agentic behavior**—all aimed at ensuring our AI systems are effective, explainable, and robust.

We believe that rigorous evaluation is the foundation of responsible AI. This role offers the opportunity to push the boundaries of **auto-evaluation, LLM-as-a-judge**, and **agentic evaluation methodologies**, influencing how AI systems are measured and improved at scale.

**About the Role**:
As an **Applied Scientist I**, you will:

- ** Design and Conduct Evaluations**: Develop and execute evaluation pipelines for LLMs and agentic systems, assessing reasoning, factual accuracy, and alignment.
- ** Automate and Scale**: Build tools and frameworks for **automatic evaluation**, including synthetic dataset creation, LLM-as-a-judge workflows, and continuous benchmarking systems.
- ** Collaborate and Translate**: Partner with applied scientists, ML engineers, and product managers to translate evaluation results into model improvements and product insights.
- ** Research and Experiment**: Prototype new evaluation metrics, contribute to internal reports, and support publications or presentations on evaluation methods.
- ** Champion Best Practices**: Promote reproducibility, transparency, and ethical AI evaluation within the team and broader organization.

**About You**:
You’re a great fit for this role if your background includes:

- Research or hands-on experience with **large language models, NLP evaluation, or agent-based AI systems**.
- Strong understanding of **LLM performance measurement**, prompt evaluation, and reliability testing.
- Proficiency in **Python** and familiarity with ML libraries such as **PyTorch**, **Transformers**, and **LangChain**.
- Comfort with experimental design, data analysis, and communicating technical findings clearly.

**Preferred Qualifications**:

- Experience with **LLM evaluation frameworks** (e.g., OpenAI Evals, HELM, LM Harness, or custom auto-eval tools).
- Familiarity with **retrieval-augmented generation (RAG)**, **tool-using agents**, or **agentic evaluation** methodologies.
- Experience in **cloud-based ML development** (AWS, Azure, or GCP).
- Record of **publications or preprints** in top-tier venues (e.g., NeurIPS, ACL, EMNLP, ICLR) or equivalent research contributions.
- Interest in **Responsible AI**, fairness, and interpretability research.

LI-AB3

**What’s in it For You?**
- ** Hybrid Work Model**: We’ve adopted a flexible hybrid working environment (2-3 days a week in the office depending on the role) for our office-based roles while delivering a seamless experience that is digitally and physically connected.
- ** Flexibility & Work-Life Balance**: Flex My Way is a set of supportive workplace policies designed to help manage personal and professional responsibilities, whether caring for family, giving back to the community, or finding time to refresh and reset. This builds upon our flexible work arrangements, including work from anywhere for up to 8 weeks per year, empowering employees to achieve a better work-life balance.
- ** Career Development and Growth**: By fostering a culture of continuous learning and skill development, we prepare our talent to tackle tomorrow’s challenges and deliver real-world solutions. Our Grow My Way programming and skills-first approach ensures you have the tools and knowledge to grow, lead, and thrive in an AI-enabled future.
- ** Industry Competitive Benefits**: We offer comprehensive benefit plans to include flexible vacation, two company-wide Mental Health Days off, access to the Headspace app, retirement savings, tuition reimbursement, employee incentive programs, and resources for mental, physical, and financial wellbeing.
- ** Culture**: Globally recognized, award-winning reputation for inclusion and belonging, flexibility, work-life balance, and more. We live by our values: Obsess over our Customers, Compete to Win, Challenge (Y)our Thinking, Act Fast / Learn Fast, and Stronger Together.
- ** Social Impact**: Make an impact in your community with our Social Impact Institute. We offer employees two paid volunteer days off annually and opportunities to get involved with pro-bono consulting projects and Environmental, Social, and Governance (ESG) initiatives.
- ** Making a Real-World Impact**: We are one of the few companies glo