Defining Open-Vocabulary Segmentation: Problem Setup, Baseline, and the Uni-OVSeg Framework

In this section, we define open-vocabulary segmentation as the task of segmenting images into masks linked to semantic categories not seen during training. We then introduce a baseline approach and present our proposed Uni-OVSeg framework, which tackles the challenge of segmenting novel categories using natural language representations of test categories.


This content originally appeared on HackerNoon and was authored by Segmentation

:::info Authors:

(1) Zhaoqing Wang, The University of Sydney and AI2Robotics;

(2) Xiaobo Xia, The University of Sydney;

(3) Ziye Chen, The University of Melbourne;

(4) Xiao He, AI2Robotics;

(5) Yandong Guo, AI2Robotics;

(6) Mingming Gong, The University of Melbourne and Mohamed bin Zayed University of Artificial Intelligence;

(7) Tongliang Liu, The University of Sydney.

:::

Abstract and 1. Introduction

2. Related works

3. Method and 3.1. Problem definition

3.2. Baseline and 3.3. Uni-OVSeg framework

4. Experiments

4.1. Implementation details

4.2. Main results

4.3. Ablation study

5. Conclusion

6. Broader impacts and References

\ A. Framework details

B. Promptable segmentation

C. Visualisation

3. Method

In this section, we first define the problem of openvocabulary segmentation in Sec. 3.1. We then introduce a straightforward baseline in Sec. 3.2. Finally, we present our proposed Uni-OVSeg framework in Sec. 3.3, including an overview, mask generation, mask-text alignment, and open-vocabulary inference.

3.1. Problem definition

\

:::info This paper is available on arxiv under CC BY 4.0 DEED license.

:::

\


This content originally appeared on HackerNoon and was authored by Segmentation


Print Share Comment Cite Upload Translate Updates
APA

Segmentation | Sciencx (2024-11-12T22:26:37+00:00) Defining Open-Vocabulary Segmentation: Problem Setup, Baseline, and the Uni-OVSeg Framework. Retrieved from https://www.scien.cx/2024/11/12/defining-open-vocabulary-segmentation-problem-setup-baseline-and-the-uni-ovseg-framework/

MLA
" » Defining Open-Vocabulary Segmentation: Problem Setup, Baseline, and the Uni-OVSeg Framework." Segmentation | Sciencx - Tuesday November 12, 2024, https://www.scien.cx/2024/11/12/defining-open-vocabulary-segmentation-problem-setup-baseline-and-the-uni-ovseg-framework/
HARVARD
Segmentation | Sciencx Tuesday November 12, 2024 » Defining Open-Vocabulary Segmentation: Problem Setup, Baseline, and the Uni-OVSeg Framework., viewed ,<https://www.scien.cx/2024/11/12/defining-open-vocabulary-segmentation-problem-setup-baseline-and-the-uni-ovseg-framework/>
VANCOUVER
Segmentation | Sciencx - » Defining Open-Vocabulary Segmentation: Problem Setup, Baseline, and the Uni-OVSeg Framework. [Internet]. [Accessed ]. Available from: https://www.scien.cx/2024/11/12/defining-open-vocabulary-segmentation-problem-setup-baseline-and-the-uni-ovseg-framework/
CHICAGO
" » Defining Open-Vocabulary Segmentation: Problem Setup, Baseline, and the Uni-OVSeg Framework." Segmentation | Sciencx - Accessed . https://www.scien.cx/2024/11/12/defining-open-vocabulary-segmentation-problem-setup-baseline-and-the-uni-ovseg-framework/
IEEE
" » Defining Open-Vocabulary Segmentation: Problem Setup, Baseline, and the Uni-OVSeg Framework." Segmentation | Sciencx [Online]. Available: https://www.scien.cx/2024/11/12/defining-open-vocabulary-segmentation-problem-setup-baseline-and-the-uni-ovseg-framework/. [Accessed: ]
rf:citation
» Defining Open-Vocabulary Segmentation: Problem Setup, Baseline, and the Uni-OVSeg Framework | Segmentation | Sciencx | https://www.scien.cx/2024/11/12/defining-open-vocabulary-segmentation-problem-setup-baseline-and-the-uni-ovseg-framework/ |

Please log in to upload a file.




There are no updates yet.
Click the Upload button above to add an update.

You must be logged in to translate posts. Please log in or register.