Imke Grabe, Jaden Fiotto-Kaufman

Patch Explorer, 2025 Patch Explorer is an interactive interface that lets users explore and manipulate the hidden layers of diffusion models. Inspired by interpretability techniques, Patch Explorer allows users to interact with the hidden states of a model by directly manipulating its activations. The diffusion process takes place in the latent space, where images are processed as patches. Through down- and up-sampling, the number of patches representing the generated image varies over layers. A patch grid represents what an individual cross-attention head adds to the generation process, with magenta displaying positive, and cyan representing negative activation. The interface aims to convey a sense of how the network forms its hidden semantics. By inspecting the visualization, the distribution of semantics across layers becomes apparent. For example, when prompted for a ''unicorn'', the creature's shape becomes more clearly defined throughout the network. Users can traverse through the U-Net's cross-attention layers, here ordered horizontally, and their attention heads, stacked vertically, over several timesteps. In the 3D view, as shown above, the timesteps are mapped along the third axis, increasing towards the user's viewpoint. In the 2D view, as shown below, the timesteps are overlayed in an orthographic manner, which can be inspected by adjusting a slider.

After (1) providing a text prompt and a seed to generate (2) an image, users can inspect the (3) contribution of the individual cross-attention heads visualized as patch grids for a selected (4) timestep range. They can choose to (5) apply an intervention to selected patch grids.

To apply an intervention, like Scaling, the user (1) types in a factor, (2) selects the timestep range, and (3) marks the patches to apply it to. More specific features are found in the blocks of lower  - for example, by manipulating certain attention heads that activate specifically for the horn, the unicorn turns into a horse.

For more details about the interface, see the paper. Credits: The interface was developed together with Rohit Gandikota and David Bau. You can read more about the project at patch.baulab.info. Patch Explorer is built on the interpretability tool NNsight and threeJS, and is presented as part of CVPR 2025’s MIV Workshop. We thank Louie Søs Meyer and Caden Juang for helpful advice along the way.