MaRiO: Multi-agent Collaborative Reasoning via Shared Observations in MLLMs

Redmond, Nathaniel; Tito Cruz, Fidel Omar; Sharma, Devansh; Azad, Shehreen; Mitra, Sirshapan; Vyas, Shruti; Rawat, Yogesh S.

MaRiO: Multi-agent Collaborative Reasoning via Shared Observations in MLLMs

Nathaniel Redmond^*, Fidel Omar Tito Cruz^*, Devansh Sharma, Shehreen Azad, Sirshapan Mitra, Shruti Vyas, Yogesh S. Rawat

Institute of Artificial Intelligence, University of Central Florida

^*Equal contribution

NeurIPS 2026, Datasets and Benchmarks Track

Try an example Papersoon Code Dataset

Agent 1 looks down a hallway towards two armchairs and a staircase. — **Multi-agent collaborative reasoning in a shared environment,** where multiple agents observe the same environment from independent coordinate frames, requiring cross-agent correspondence before inferring spatial and geometric relations. Only the three views on the left go to the model. The cutaway is here so you can see where the agents actually are.

Agent 2 faces a staircase with a bicycle and a potted plant beside it. — **Multi-agent collaborative reasoning in a shared environment,** where multiple agents observe the same environment from independent coordinate frames, requiring cross-agent correspondence before inferring spatial and geometric relations. Only the three views on the left go to the model. The cutaway is here so you can see where the agents actually are.

Abstract

Multimodal Large Language Models (MLLMs) show strong visual perception capabilities, but existing evaluations largely focus on single-view or multi-view settings with shared camera parameters, leaving the challenge of multi-agent collaborative scenarios underexplored. In such settings, multiple agents observe a shared environment from independent viewpoints, and reliable decisions require reasoning across these independent observations of the same environment. To study this problem, we introduce MaRiO, a benchmark for evaluating multi-agent collaborative reasoning, and evaluate 23 state-of-the-art MLLMs on it. Our analysis reveals three key findings: (1) tasks governed by static perception are generally easier than tasks that require transformational event sequences; (2) most models struggle to resolve cross-agent correspondences, with errors increasing with scene complexity; and (3) models show limited ability to leverage geometric cues present in the scene. As an initial step toward addressing these challenges, we explore geometry-guided synthetic view augmentation that generates intermediate transition frames between agent observations, providing auxiliary spatial context that improves performance in certain categories. Overall, our results highlight that multi-agent collaborative reasoning remains an open challenge and an important direction for multimodal embodied systems.

Motivation

Comparison table. MaRiO is the only benchmark listed that is multi-agent, has independent camera geometry and provides nothing beyond RGB. — **Comparison of MaRiO with existing visual reasoning benchmarks.** MV and MA respectively denote Multi-view and Multi-agent. Perc. and Comp. respectively refer to static perception and compositional reasoning over geometric transformations. Cam. Geom. refers to the provision of explicit camera geometry, and Aux. Info. refers to auxiliary information beyond RGB frames.

Single view: One image, so there are no frames to match.
Multi-view: Several images, captured along one trajectory and usually shipped with poses, intrinsics or point clouds.
Multi-agent: Each agent has its own egocentric frame and no shared metadata. Matching the frames is the first half of every question.

The MaRiO benchmark

3,255questions

1,680observation sets

1,371agent pairs

309agent triplets

5tasks

23models

Input. Two or three synchronized views and a four-choice question. No camera poses, no intrinsics, no depth, no scene graphs.

Scenes. 913 real image sets from RealEstate10K and NAVI, plus 767 synthetic sets from VirtualHome, Habitat-Sim and AI2-THOR.

Distance Awareness, Occlusion and Relative Position ask for one spatial fact once the views are matched. Egocentric Motion and Navigation ask the model to compose a transformation between frames.

Three domain-expert annotators drafted and reviewed every question, and a further annotator cross-verified them. An LLM polished phrasing and drafted distractors. Every ground-truth label is human.

Loading examples…

Results

Zero-shot accuracy of 23 models on the five tasks. Human evaluators score 73.2 overall and the best model 55.7. — **Zero-shot performance of multi-agent collaborative reasoning on MaRiO.** Dist.: Distance awareness, Ego.: Egocentric motion, Nav.: Navigation, Occ.: Occlusion, and Rel.: Relative position. Model type-wise best and second-best scores are highlighted, with overall best and second-best additionally emphasized. Comp. and Static Perc. respectively refer to Compositional Reasoning over Geometric Transformations and Static Perception.

The gap. Humans score 73.2. The best model, Gemini-2.5-Pro, stops at 55.7, which is 17.5 points short. The best open model, Qwen2-VL-7B-Instruct, gets 42.0. Chance is 23.9.

By task. Navigation and Egocentric Motion sit closest to chance. Distance Awareness and Occlusion hold up best. Models still see the scene. What they cannot do is carry a direction out of one agent's frame and into another's.

Analysis

Compositional reasoning and cross-agent correspondence

Static perceptual accuracy plotted against compositional accuracy. The points fill the whole square and the fitted line is almost flat. — **(a)** Static perceptual accuracy barely predicts compositional accuracy, slope 0.15

Accuracy against shared anchor density. Every task rises to a peak at the six to nine bin and falls at ten or more. — **(b)** Accuracy peaks at six to nine shared anchors, then falls away

Five models compared with all views present, one view missing and all views missing. On the compositional tasks the three bars are nearly level for the open-source models. — **(c)** Removing agent views leaves compositional accuracy almost unchanged

Six models, two-agent against three-agent accuracy. The three-agent bar is lower for almost every model and task. — **(d)** A third agent lowers accuracy across almost every model and task

Geometric cues and the human comparison

Left, how often each model answer changes under mirroring, with bias ratios close to one half. Right, directional navigation questions shift further than obstacle questions. — **(a)** Mirroring a scene changes 22 to 32 percent of answers

Accuracy change between high and low depth edge density for six models, within the simple, moderate and complex strata. The effects scatter around zero. — **(b)** Denser depth structure does not help, and most models do worse

Left, the 23 incorrect human responses by type, with 13 of them direction confusions. Right, human against Gemini-2.5-Pro correctness on the same 81 questions. — **(c)** Humans reach 73.2 percent, missing mostly on direction

Geometry-guided view augmentation

1 Agent views
2 Point cloud
3 Interpolated frames
4 Novel views

Geometry-guided view synthesis, in four stages.

Four models before and after adding synthetic views, with the change on each task. Qwen2.5-VL-7B gains 1.1 overall, LLaVA-Video-7B and VLM-3R-7B are roughly flat, and VST-7B-RL drops 1.9. — **Performance of synthetic view augmentations.** Improvements after adding synthetic views and best overall gain are highlighted.

Setup. A diffusion video model turns the rendered in-between frames into novel views. They go in as extra context for the same question, and no new agent is added.

Effect. Qwen2.5-VL-7B gains 1.1 overall, with 4.5 on Occlusion and 3.9 on Relative Position. Spatially trained models gain nothing or lose ground, which the paper reads as the generated frames introducing small geometric inconsistencies. Twelve frames is the best setting, and cropping real frames instead does not reproduce the gain, so what helps is the new viewpoint and not the extra pixels.