---
title: "视觉相似性的多重意义：一种文本提示驱动的图像感知度量"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.18237v1"
permalink: "https://feed.astralor.com/items/itm_1f9c39a456e16b8109e11e27"
published: "2026-07-20T17:59:51.000Z"
score: 79
channel: "ai"
tags: ["TPIPS","文本引导检索","视觉相似性","视觉语言模型"]
---

# 视觉相似性的多重意义：一种文本提示驱动的图像感知度量

## AI 点评

> 我认为这篇论文切中了视觉感知度量的一个核心痛点——缺乏语义条件化能力。TPIPS让用户用自然语言指定比较维度，这比固定度量灵活得多。对于做图像检索、生成模型评估的研究者来说，这很可能成为实用的新工具。开源资源也降低了复现门槛，值得关注。

## 速览

人类对图像相似性的判断高度依赖上下文——两幅图可能形状相似但颜色不同，而现有度量只给出单一标量值，无法根据不同语义方面进行条件化。本文发布了大规模图像三元组人类相似性判断数据集，每个三元组标注了多个自由形式的语义相似性方面。对前沿视觉语言模型（VLM）的基准测试发现，它们与人类共识存在显著差距。作者基于数据微调VLM，提出文本提示驱动的图像感知相似度（TPIPS），能够根据指定文本提示捕捉多种视觉相似性含义。实验表明TPIPS更贴近人类感知，且泛化到分布外数据。该方法在文本引导检索、组合搜索和生成模型细粒度评估中展现新能力。

## 关键点

- 构建了大规模图像三元组人类相似性判断数据集，每个三元组包含多个自由形式语义方面的标注。
- 对主流视觉语言模型（VLM）进行基准测试，发现其与人类共识有显著差距。
- 微调VLM得到TPIPS度量，能根据文本提示条件化地捕捉多种视觉相似性维度。
- TPIPS在文本引导检索、组合搜索和生成模型细粒度评估中表现优异。

## 全文

Human visual similarity judgments are context-dependent. For example, two images may be similar in shape but distinct in color. Existing perceptual similarity metrics, however, collapse these nuances into a single scalar value, offering no mechanism to condition on specific aspects. To bridge this gap, we introduce a large-scale dataset of human similarity judgments over image triplets, where each triplet is annotated across multiple, free-form semantic aspects of similarity. Benchmarking a broad range of frontier vision-language models (VLMs) reveals a considerable performance gap compared to human annotators' consensus. Leveraging our data, we fine-tune a VLM to produce our Text-Prompted Image Perceptual Similarity (TPIPS) metric, capturing multiple senses of visual similarity depending on the specified text prompt. We demonstrate that TPIPS aligns more closely with human perception and generalizes reliably beyond the training distribution. Finally, we show that TPIPS unlocks new capabilities in text-guided retrieval, compositional search, and the fine-grained evaluation of generative models. Our code, data, and trained models are at https://peterwang512.github.io/TPIPS

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_1f9c39a456e16b8109e11e27