ParVL: Parallel Scaling and Expandable Compute Allocation for Multimodal LLMs

ParVL (Parallel Vision-Language) is a new scaling framework for multimodal LLMs. Instead of expanding parameters or sequential inference depth, ParVL enables parallel compute scaling by reusing shared ViT and LLM backbone parameters across multiple vision and language branches, avoiding significant memory and latency overhead. The central question is how to allocate additional shared-backbone compute between vision and language under a fixed backbone parameter budget. The researchers instantiate each parallel stream with branch-specific prefix parameters over a shared backbone and perform full-parameter supervised fine-tuning on roughly 13B tokens. Experiments reveal a clear compute-allocation trade-off between the ViT encoder and LLM decoder, showing that ParVL improves overall multimodal performance over same-recipe single-branch baselines, and the optimal vision-language allocation varies across tasks. Code is publicly available for reproduction and further exploration.