How Concept Frequency Affects AI Image Accuracy

The study shows that text-to-image (T2I) model accuracy—specifically Stable Diffusion—improves for concepts that appear more frequently in its training data. Using 360 public figures randomly sampled from Wikidata, researchers found a consistent log-linear trend: the more often a person appeared in LAION-Aesthetic captions, the more accurately the model could generate their likeness. Human evaluators confirmed these findings, supporting concept frequency as a strong predictor of model performance.


This content originally appeared on HackerNoon and was authored by The FewShot Prompting Publication

Abstract and 1. Introduction

2 Concepts in Pretraining Data and Quantifying Frequency

3 Comparing Pretraining Frequency & “Zero-Shot” Performance and 3.1 Experimental Setup

3.2 Result: Pretraining Frequency is Predictive of “Zero-Shot” Performance

4 Stress-Testing the Concept Frequency-Performance Scaling Trend and 4.1 Controlling for Similar Samples in Pretraining and Downstream Data

4.2 Testing Generalization to Purely Synthetic Concept and Data Distributions

5 Additional Insights from Pretraining Concept Frequencies

6 Testing the Tail: Let It Wag!

7 Related Work

8 Conclusions and Open Problems, Acknowledgements, and References

Part I

Appendix

A. Concept Frequency is Predictive of Performance Across Prompting Strategies

B. Concept Frequency is Predictive of Performance Across Retrieval Metrics

C. Concept Frequency is Predictive of Performance for T2I Models

D. Concept Frequency is Predictive of Performance across Concepts only from Image and Text Domains

E. Experimental Details

F. Why and How Do We Use RAM++?

G. Details about Misalignment Degree Results

H. T2I Models: Evaluation

I. Classification Results: Let It Wag!

C Concept Frequency is Predictive of Performance for T2I Models

We extend the results from Fig. 3 with Figs. 11 to 15. As with Fig. 3, due to the high concept frequency, the scaling trend is weaker. Furthermore, we do see inconsistency in the trends for the human-rated scores retrieved from HEIM [71], hence we perform a small scale human evaluation to check them.

\ Given the societal relevance [23], we decided to test Stable Diffusion [96] (v1.4) on generating public figures. We scraped 50,000 people from the “20230123-all” Wikidata JSON dump by filtering for entities listed as “human” [8], and scraped a reference image for the human study for each person if an image was available. After computing concept frequency from LAION-Aesthetics text captions (using suffix array [70]), we found that ≈10,000 people were present in the pretraining dataset. Note that to ensure the people’s names were treated as separate words, we computed frequency for strings of the format “ {entity} ”. We then randomly sample 360 people (for which a reference image was available) normalized by frequency [22] for the human study. For generating images with Stable Diffusion, we used the prompt “headshot of {entity}”, in order to specify to the model that “{entity}” is referring to the person named “{entity}” [50].

\ We assessed image-text alignment with a human study with 6 participants, where each participant was assigned 72 samples; for consistency, of the 360 total samples, we ensured 10% were assigned to 3 participants. Provided with a reference image, the participants were asked if the sample accurately depicts the prompt. Three choices were provided: “Yes” (score=1.), “Somewhat” (score=0.5), and “No” (score=0.). Accuracy was computed by averaging the scores.

\ As can be seen in Fig. 16, we observe a log-linear trend between concept frequency and zero-shot performance. Thus, we observe that the log-linear trend between concept frequency and zero-shot performance consistently holds even for T2I models.

\ Figure 11: Log-linear relationships between concept frequency and T2I Max aesthetic scores. Across all tested models pretrained on the LAION-Aesthetics dataset, we observe a consistent linear relationship between T2I zero-shot performance on a concept and the log-scaled concept pretraining frequency.

\ Figure 12: Log-linear relationships between concept frequency and T2I human aesthetic scores. Across all tested models pretrained on the LAION-Aesthetics dataset, we observe a consistent linear relationship between T2I zero-shot performance on a concept and the log-scaled concept pretraining frequency.

\ Figure 13: Log-linear relationships between concept frequency and T2I human alignment scores. Across all tested models pretrained on the LAION-Aesthetics dataset, we observe a consistent linear relationship between T2I zero-shot performance on a concept and the log-scaled concept pretraining frequency.

\ Figure 14: Log-linear relationships between concept frequency and T2I Avg. CLIP scores. Across all tested models pretrained on the LAION-Aesthetics dataset, we observe a consistent linear relationship between T2I zero-shot performance on a concept and the log-scaled concept pretraining frequency.

\ Figure 15: Log-linear relationships between concept frequency and T2I Max CLIP scores. Across all tested models pretrained on the LAION-Aesthetics dataset, we observe a consistent linear relationship between T2I zero-shot performance on a concept and the log-scaled concept pretraining frequency.

\ Figure 16: Log-linear relationship between concept frequency and T2I human evaluation for text-image alignment for people concepts. We observe a consistent linear relationship between T2I zero-shot performance on a concept and the log-scaled concept pretraining frequency.

\

:::info Authors:

(1) Vishaal Udandarao, Tubingen AI Center, University of Tubingen, University of Cambridge, and equal contribution;

(2) Ameya Prabhu, Tubingen AI Center, University of Tubingen, University of Oxford, and equal contribution;

(3) Adhiraj Ghosh, Tubingen AI Center, University of Tubingen;

(4) Yash Sharma, Tubingen AI Center, University of Tubingen;

(5) Philip H.S. Torr, University of Oxford;

(6) Adel Bibi, University of Oxford;

(7) Samuel Albanie, University of Cambridge and equal advising, order decided by a coin flip;

(8) Matthias Bethge, Tubingen AI Center, University of Tubingen and equal advising, order decided by a coin flip.

:::


:::info This paper is available on arxiv under CC BY 4.0 DEED license.

:::

\


This content originally appeared on HackerNoon and was authored by The FewShot Prompting Publication


Print Share Comment Cite Upload Translate Updates
APA

The FewShot Prompting Publication | Sciencx (2025-07-09T11:00:04+00:00) How Concept Frequency Affects AI Image Accuracy. Retrieved from https://www.scien.cx/2025/07/09/how-concept-frequency-affects-ai-image-accuracy/

MLA
" » How Concept Frequency Affects AI Image Accuracy." The FewShot Prompting Publication | Sciencx - Wednesday July 9, 2025, https://www.scien.cx/2025/07/09/how-concept-frequency-affects-ai-image-accuracy/
HARVARD
The FewShot Prompting Publication | Sciencx Wednesday July 9, 2025 » How Concept Frequency Affects AI Image Accuracy., viewed ,<https://www.scien.cx/2025/07/09/how-concept-frequency-affects-ai-image-accuracy/>
VANCOUVER
The FewShot Prompting Publication | Sciencx - » How Concept Frequency Affects AI Image Accuracy. [Internet]. [Accessed ]. Available from: https://www.scien.cx/2025/07/09/how-concept-frequency-affects-ai-image-accuracy/
CHICAGO
" » How Concept Frequency Affects AI Image Accuracy." The FewShot Prompting Publication | Sciencx - Accessed . https://www.scien.cx/2025/07/09/how-concept-frequency-affects-ai-image-accuracy/
IEEE
" » How Concept Frequency Affects AI Image Accuracy." The FewShot Prompting Publication | Sciencx [Online]. Available: https://www.scien.cx/2025/07/09/how-concept-frequency-affects-ai-image-accuracy/. [Accessed: ]
rf:citation
» How Concept Frequency Affects AI Image Accuracy | The FewShot Prompting Publication | Sciencx | https://www.scien.cx/2025/07/09/how-concept-frequency-affects-ai-image-accuracy/ |

Please log in to upload a file.




There are no updates yet.
Click the Upload button above to add an update.

You must be logged in to translate posts. Please log in or register.