---
title: "Segment What It Can Draw: Querying a Text-to-Image Generator with Its Own Images for Skin Lesion Segmentation"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2611.07061v1"
permalink: "https://feed.astralor.com/items/itm_9381e9f32bdaa8726f33b241"
published: "2026-11-04T19:20:08.000Z"
score: 58
channel: "tech"
tags: ["推理效率","Segment What","Can Draw","Querying","Text-to-Image Generator","Its Own Images","Skin Lesion Segmentation","Skin"]
---

# Segment What It Can Draw: Querying a Text-to-Image Generator with Its Own Images for Skin Lesion Segmentation

## At a glance

Skin lesion segmentation without annotated masks requires the target to be specified through other cues. Existing approaches rely on dermoscopy-specific rules, annotation-derived model priors, or user-selected regions. …

## Key points

- Skin lesion segmentation without annotated masks requires the target to be specified through other cues.
- Existing approaches rely on dermoscopy-specific rules, annotation-derived model priors, or user-selected regions.
- A general text-to-image diffusion transformer offers a text interface to the target, but its concept-word attention is unreliable on dermoscopy images: in the model's attention-output space, the word-derived skin-to-les…

## Full text

Skin lesion segmentation without annotated masks requires the target to be specified through other cues. Existing approaches rely on dermoscopy-specific rules, annotation-derived model priors, or user-selected regions. A general text-to-image diffusion transformer offers a text interface to the target, but its concept-word attention is unreliable on dermoscopy images: in the model's attention-output space, the word-derived skin-to-lesion direction is poorly aligned with the direction separating real skin from real lesions. We show that the direction derived from generated healthy-skin and lesion images is substantially better aligned. Based on this finding, we propose GENIQ, a training-free method that constructs initial skin and lesion prototypes from generated images and refines them using the most skin-like and lesion-like patches from unlabelled dermoscopy images. It scores test patches along the refined direction and applies a gated threshold to suppress contextual spill-over around small lesions. Without inference-time annotations or additional models, GENIQ achieves Dice scores of 0.85-0.88 on three dermoscopy test sets. It outperforms zero-shot and word-based attention methods on all three sets, and a hand-designed dermoscopy pipeline in Dice and IoU on two.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_9381e9f32bdaa8726f33b241