How to Measure the Quality of AIGC Services

The article breaks down how to evaluate AIGC service quality using both objective metrics like latency and subjective client feedback, calibrated for fairness.


This content originally appeared on HackerNoon and was authored by EScholar: Electronic Academic Papers for Scholars

Abstract and 1. Introduction

1.1 Background

1.2 Motivation

1.3 Our Work and Contributions and 1.4 Organization

  1. Related Work

    2.1 Mobile AIGC and Its QoE Modeling

    2.2 Blockchain for Mobile Networks

  2. Preliminaries

  3. Prosecutor Design

    4.1 Architecture Overview

    4.2 Reputation Roll-up

    4.3 Duplex Transfer Channel

  4. OS2a: Objective Service Assessment for Mobile AIGC

    5.1 Inspiration from DCM

    5.2 Objective Quality of the Service Process

    5.3 Subjective Experience of AIGC Outputs

  5. OS2A on Prosecutor: Two-Phase Interaction for Mobile AIGC

    6.1 MASP Selection by Reputation

    6.2 Contract Theoretic Payment Scheme

  6. Implementation and Evaluation

    7.1 Implementation and Experimental Setup

    7.2 Prosecutor Performance Evaluation

    7.3 Investigation of Functional Goals

    7.4 Security Analysis

  7. Conclusion and References

5.2 Objective Quality of the Service Process

The objective quality of the AIGC service process can be evaluated by various KPIs from different perspectives. Service latency is one of the most critical concerns because, 8 given the immature AIGC governance, the currently common practice is that the client who first publishes one AIGC output owns the copyright [46]. Consequently, clients will always expect a service latency as small as possible to prevent similar ideas from being published by other mobile neighbors. Without loss of generality, in this paper, we take AIGC service latency as an example to showcase the OS2AO calculation.

\

\

\ 5.2.1 Modeling of Channel Establishment

\

\

\ Fig. 5: The illustration of transaction pool and the fitting of Ethereum transaction fee.

\

\ 5.2.2 Generalization to Other KPIs

\

5.3 Subjective Experience of AIGC Outputs

Besides the objective KPIs, clients also care about whether the generated AIGC outputs can meet their personal requirements, which we coin as the experiences of the AIGC service outcome. Considering the subjectivity in artwork assessment, clients evaluate the AIGC outputs using their own standards and preferences. Nonetheless, regardless of which kind of model is used, the final output should be converted to a boolean judgment, whose value is 0 or 1. Value 1 means that the client receives a satisfying AIGC output. In contrast, value 0 indicates that the received output suffers from flaws, e.g., low resolution, distortion, and mismatched art style. Then, an intuitive way is to average the judgment items from all the clients and calculate the reputation score for every MASP. However, recall that these clients hold different levels of strictness. We should calibrate the judgment from numerous heterogeneous clients and generate a fair overall reputation for each MASP. To reach this goal, we use WMSL [16] to calibrate the judgment, as shown in Algorithm 2.

\ 5.3.1 Local Opinion

\ \

\ \ 5.3.2 Reference Opinion and Calibration

\ \

\ \ \

\ \ \

\ \ \

\ \ \

\ \ \

\ \ \

\ \ 5.3.3 Opinion Fusion and Reputation

\ \

\ \ \ Fig. 6: The perceivable experience of AIGC output quality. Images (a) and (b) are trained on AnyLoRA-Checkpoint model using the same prompt: Arcane style, 1 girl, pink hair, long hair, one braid, white shirt, coat, yellow eyes, looking at viewer, city street. However, the model for Image (b) is fine-tuned by Arcane Style LoRA. Image (c) shows some references of Arcane style from Google. This example is from [55].

\ \ \

\ \ 5.3.4 Discussion: Rationale of Reputation

\ \

\

:::info Authors:

(1) Yinqiu Liu, School of Computer Science and Engineering, Nanyang Technological University, Singapore (yinqiu001@e.ntu.edu.sg);

(2) Hongyang Du, School of Computer Science and Engineering, Nanyang Technological University, Singapore (hongyang001@e.ntu.edu.sg);

(3) Dusit Niyato, School of Computer Science and Engineering, Nanyang Technological University, Singapore (dniyato@ntu.edu.sg);

(4) Jiawen Kang, School of Automation, Guangdong University of Technology, China (kavinkang@gdut.edu.cn);

(5) Zehui Xiong, Pillar of Information Systems Technology and Design, Singapore University of Technology and Design, Singapore (zehuixiong@sutd.edu.sg);

(6) Abbas Jamalipour, School of Electrical and Information Engineering, University of Sydney, Australia (a.jamalipour@ieee.org);

(7) Xuemin (Sherman) Shen, Department of Electrical and Computer Engineering, University of Waterloo, Canada (sshen@uwaterloo.ca).

:::


:::info This paper is available on arxiv under CC BY 4.0 DEED license.

:::

[3] Data available at http://xblock.pro/#/dataset/14


This content originally appeared on HackerNoon and was authored by EScholar: Electronic Academic Papers for Scholars


Print Share Comment Cite Upload Translate Updates
APA

EScholar: Electronic Academic Papers for Scholars | Sciencx (2025-06-25T11:00:16+00:00) How to Measure the Quality of AIGC Services. Retrieved from https://www.scien.cx/2025/06/25/how-to-measure-the-quality-of-aigc-services/

MLA
" » How to Measure the Quality of AIGC Services." EScholar: Electronic Academic Papers for Scholars | Sciencx - Wednesday June 25, 2025, https://www.scien.cx/2025/06/25/how-to-measure-the-quality-of-aigc-services/
HARVARD
EScholar: Electronic Academic Papers for Scholars | Sciencx Wednesday June 25, 2025 » How to Measure the Quality of AIGC Services., viewed ,<https://www.scien.cx/2025/06/25/how-to-measure-the-quality-of-aigc-services/>
VANCOUVER
EScholar: Electronic Academic Papers for Scholars | Sciencx - » How to Measure the Quality of AIGC Services. [Internet]. [Accessed ]. Available from: https://www.scien.cx/2025/06/25/how-to-measure-the-quality-of-aigc-services/
CHICAGO
" » How to Measure the Quality of AIGC Services." EScholar: Electronic Academic Papers for Scholars | Sciencx - Accessed . https://www.scien.cx/2025/06/25/how-to-measure-the-quality-of-aigc-services/
IEEE
" » How to Measure the Quality of AIGC Services." EScholar: Electronic Academic Papers for Scholars | Sciencx [Online]. Available: https://www.scien.cx/2025/06/25/how-to-measure-the-quality-of-aigc-services/. [Accessed: ]
rf:citation
» How to Measure the Quality of AIGC Services | EScholar: Electronic Academic Papers for Scholars | Sciencx | https://www.scien.cx/2025/06/25/how-to-measure-the-quality-of-aigc-services/ |

Please log in to upload a file.




There are no updates yet.
Click the Upload button above to add an update.

You must be logged in to translate posts. Please log in or register.