---
title: "τ-bench-LILT: Evaluating Conversational Agents on Realistic Multilingual Tool-Use Tasks"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2611.07203v1"
permalink: "https://feed.astralor.com/items/itm_7793e9701411634b1948ee29"
published: "2026-11-04T22:47:38.000Z"
score: 55
channel: "tech"
tags: ["AI Agent","LILT","Evaluating Conversational Agents","Realistic Multilingual Tool-Use","Tasks Frontier","English","Frontier"]
---

# τ-bench-LILT: Evaluating Conversational Agents on Realistic Multilingual Tool-Use Tasks

## 速览

Frontier evaluations of tool-using conversational agents are conducted almost exclusively in English, leaving a critical gap for organizations deploying agents in other languages and regions. We present $τ$-bench-LILT, …

## 关键点

- Frontier evaluations of tool-using conversational agents are conducted almost exclusively in English, leaving a critical gap for organizations deploying agents in other languages and regions.
- We present $τ$-bench-LILT, a multilingual extension of ${τ^3}$-bench covering German, Korean, Turkish, and Farsi in the airline, retail, telecom, and banking domains.
- We natively localize every layer of the benchmark: customer databases are populated with culturally appropriate records, user-simulator prompts are refined iteratively through contrastive annotation by native-language e…

## 全文

Frontier evaluations of tool-using conversational agents are conducted almost exclusively in English, leaving a critical gap for organizations deploying agents in other languages and regions. We present $τ$-bench-LILT, a multilingual extension of ${τ^3}$-bench covering German, Korean, Turkish, and Farsi in the airline, retail, telecom, and banking domains. We natively localize every layer of the benchmark: customer databases are populated with culturally appropriate records, user-simulator prompts are refined iteratively through contrastive annotation by native-language experts, and tasks are updated according to the localized database and to reflect realistic tasks in the target region. Evaluation across eight agent models shows the largest drops compared to English in the Korean and Farsi retail domains, by up to 61 points in pass^4. Code and data released at https://github.com/lilt/tau-bench-lilt.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_7793e9701411634b1948ee29