From Images to Programs: A Denoising Diffusion Method for Inverse Graphics

This article presents a novel method for program synthesis using denoising diffusion models on syntax trees.


This content originally appeared on HackerNoon and was authored by Photosynthesis Technology: It's not just for plants!

Abstract and 1. Introduction

  1. Background & Related Work

  2. Method

    3.1 Sampling Small Mutations

    3.2 Policy

    3.3 Value Network & Search

    3.4 Architecture

  3. Experiments

    4.1 Environments

    4.2 Baselines

    4.3 Ablations

  4. Conclusion, Acknowledgments and Disclosure of Funding, and References

    \

Appendix

A. Mutation Algorithm

B. Context-Free Grammars

C. Sketch Simulation

D. Complexity Filtering

E. Tree Path Algorithm

F. Implementation Details

3 Method

The main idea behind our method is to develop a form of denoising diffusion models analogous to image diffusion models for syntax trees.

\ Consider the example task from Ellis et al. [11] of generating a constructive solid geometry (CSG2D) program from an image. In CSG2D, we can combine simple primitives like circles and quadrilaterals using boolean operations like addition and subtraction to create more complex shapes, with the context-free grammar (CFG),

\

\ In the following sections, we will first describe how “noise” is added to syntax trees. Then, we will detail how we train a neural network to reverse this noise. Finally, we will describe how we use this neural network for search.

\

:::info Authors:

(1) Shreyas Kapur, University of California, Berkeley (srkp@cs.berkeley.edu);

(2) Erik Jenner, University of California, Berkeley (jenner@cs.berkeley.edu);

(3) Stuart Russell, University of California, Berkeley (russell@cs.berkeley.edu).

:::


:::info This paper is available on arxiv under CC BY-SA 4.0 DEED license.

:::

\


This content originally appeared on HackerNoon and was authored by Photosynthesis Technology: It's not just for plants!


Print Share Comment Cite Upload Translate Updates
APA

Photosynthesis Technology: It's not just for plants! | Sciencx (2025-09-24T15:00:03+00:00) From Images to Programs: A Denoising Diffusion Method for Inverse Graphics. Retrieved from https://www.scien.cx/2025/09/24/from-images-to-programs-a-denoising-diffusion-method-for-inverse-graphics/

MLA
" » From Images to Programs: A Denoising Diffusion Method for Inverse Graphics." Photosynthesis Technology: It's not just for plants! | Sciencx - Wednesday September 24, 2025, https://www.scien.cx/2025/09/24/from-images-to-programs-a-denoising-diffusion-method-for-inverse-graphics/
HARVARD
Photosynthesis Technology: It's not just for plants! | Sciencx Wednesday September 24, 2025 » From Images to Programs: A Denoising Diffusion Method for Inverse Graphics., viewed ,<https://www.scien.cx/2025/09/24/from-images-to-programs-a-denoising-diffusion-method-for-inverse-graphics/>
VANCOUVER
Photosynthesis Technology: It's not just for plants! | Sciencx - » From Images to Programs: A Denoising Diffusion Method for Inverse Graphics. [Internet]. [Accessed ]. Available from: https://www.scien.cx/2025/09/24/from-images-to-programs-a-denoising-diffusion-method-for-inverse-graphics/
CHICAGO
" » From Images to Programs: A Denoising Diffusion Method for Inverse Graphics." Photosynthesis Technology: It's not just for plants! | Sciencx - Accessed . https://www.scien.cx/2025/09/24/from-images-to-programs-a-denoising-diffusion-method-for-inverse-graphics/
IEEE
" » From Images to Programs: A Denoising Diffusion Method for Inverse Graphics." Photosynthesis Technology: It's not just for plants! | Sciencx [Online]. Available: https://www.scien.cx/2025/09/24/from-images-to-programs-a-denoising-diffusion-method-for-inverse-graphics/. [Accessed: ]
rf:citation
» From Images to Programs: A Denoising Diffusion Method for Inverse Graphics | Photosynthesis Technology: It's not just for plants! | Sciencx | https://www.scien.cx/2025/09/24/from-images-to-programs-a-denoising-diffusion-method-for-inverse-graphics/ |

Please log in to upload a file.




There are no updates yet.
Click the Upload button above to add an update.

You must be logged in to translate posts. Please log in or register.