This content originally appeared on HackerNoon and was authored by Instancing
Table of Links
Related Work
Methodology
3.1. Preliminaries and Notations
3.2. Relations between Attention-based VPG and MIL
3.3. MIVPG for Multiple Visual Inputs
3.4. Unveiling Instance Correlation in MIVPG for Enhanced Multi-instance Scenarios
Experiments and 4.1. General Setup
4.2. Scenario 1: Samples with Single Image
4.3. Scenario 2: Samples with Multiple Images, with Each Image as a General Embedding
\ Supplementary Material
A. Detailed Architecture of QFormer
3.3. MIVPG for Multiple Visual Inputs
\ When a sample comprises multiple images, it is imperative to consider MIL feature aggregation from different perspectives. In the context of individual images, each image can be treated as a ’bag,’ and each patch within the image as an ’instance.’ From the sample’s perspective, each sample can also be regarded as a ’bag,’ with each image within the sample as an ’instance.’ When a sample contains only a single image, we can focus primarily on the former perspective since the latter perspective involves a single instance per bag. However, in a more general context, it is essential to adopt a hierarchical approach when considering the utilization of MIL for feature aggregation. Without loss of generality, we now consider the input of the MIVPG to be a bag B containing multiple instances. Hence, the cross-attention can be expressed as Attention(Q = q, K = B, V = B).
\
:::info Authors:
(1) Wenliang Zhong, The University of Texas at Arlington (wxz9204@mavs.uta.edu);
(2) Wenyi Wu, Amazon (wenyiwu@amazon.com);
(3) Qi Li, Amazon (qlimz@amazon.com);
(4) Rob Barton, Amazon (rab@amazon.com);
(5) Boxin Du, Amazon (boxin@amazon.com);
(6) Shioulin Sam, Amazon (shioulin@amazon.com);
(7) Karim Bouyarmane, Amazon (bouykari@amazon.com);
(8) Ismail Tutar, Amazon (ismailt@amazon.com);
(9) Junzhou Huang, The University of Texas at Arlington (jzhuang@uta.edu).
:::
:::info This paper is available on arxiv under CC by 4.0 Deed (Attribution 4.0 International) license.
:::
\
This content originally appeared on HackerNoon and was authored by Instancing
Instancing | Sciencx (2025-11-15T02:28:16+00:00) Multimodal Fusion: MIVPG’s Hierarchical MIL Approach for Multi-Image Samples. Retrieved from https://www.scien.cx/2025/11/15/multimodal-fusion-mivpgs-hierarchical-mil-approach-for-multi-image-samples/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.