---
title: "Benchmarking decision models against generative large language models on medical multiple-choice questions"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2610.38448v1"
permalink: "https://feed.astralor.com/items/itm_5f90805c36c579c91493e3db"
published: "2026-10-28T08:25:35.000Z"
score: 53
channel: "tech"
tags: ["LLM","Benchmarking","Medical","MedQA","Accuracy","Among","Mercury Decide","USD"]
---

# Benchmarking decision models against generative large language models on medical multiple-choice questions

## 速览

Medical workflows often require selecting among predefined options. We compared 13 decision models with 15 generative large language model configurations, including different reasoning settings, using 1,273 four-option …

## 关键点

- Medical workflows often require selecting among predefined options.
- We compared 13 decision models with 15 generative large language model configurations, including different reasoning settings, using 1,273 four-option MedQA questions under text-only conditions.
- Accuracy ranged from 67.71% to 89.08% for decision models and from 78.32% to 96.86% for generative configurations.

## 全文

Medical workflows often require selecting among predefined options. We compared 13 decision models with 15 generative large language model configurations, including different reasoning settings, using 1,273 four-option MedQA questions under text-only conditions. Accuracy ranged from 67.71% to 89.08% for decision models and from 78.32% to 96.86% for generative configurations. Among the tested configurations, Mercury Decide had the lowest mean final-attempt cost (USD 0.0069 per 1,000 questions), while Clef Omni had the shortest median request duration (0.291 seconds). Decision-model probabilities distinguished correct from incorrect answers, but error rates among answers assigned probabilities of at least 0.90 ranged from 1.71% to 12.77%. Answer agreement across three runs ranged from 92.30% to 100%, while probability repeatability varied. Decision models offered resource advantages alongside lower peak accuracy; selecting models for medical workflows requires considering probability quality and repeatability together with accuracy and the resources needed to detect and correct errors.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_5f90805c36c579c91493e3db