d) Composition Model and Training Criteria Figure 3: Possible design choices for the main d) Composition Model and Training Criteria Figure 3: Possible design choices for the main components identified in Fig. 2. Modules can take as input the input image I, features from the input image $f ( I )$ , the sprites S, the transformed sprites $\bar { S } ^ { I }$ and the predicted sprite probabilities $p ^ { I }$ . (a) The Sprite Generation Module ( ) can learn the sprites directly as learnable parameters (Pixels), generate them from learnable latent variables with a multi-layer perceptron (MLP) or a UNet architecture. (b) The Transformation Module ( ) parameters can be learned with a shared or sprite-specific network, and with diferent curriculum learning strategies. (c) The Decision Module ( ) can select sprites leading to the minimum reconstruction error (Min-Loss), or predict them using the sprites’ latent representations (Weight Prediction), or directly a linear projection (Linear Mapping), with alternative activations. (d) The Composition Model and Training Criteria ( ), where the main loss can either be the sum of the reconstruction errors obtained with all the possible sprites selection weighted by their probability $\left( \mathcal { L } _ { 0 - 1 } \right)$ or the reconstruction error with composite sprites $\left( \mathcal { L } _ { \mathrm { c o m p } } \right)$ . It can also include regularizations $( { \mathcal { L } } _ { \{ { \mathrm { f r e q } } , { \mathrm { b i n } } , { \mathrm { e m p t y } } \} } )$这张图概括 Deep Sprite-based Image Models 的整体方法流程。阅读时先看模块之间传递的训练信号，再看作者如何把目标拆成可优化的子问题。Figureure 2 · : OverviewFigure 2: Overview. We decompose all sprite-based models in four main components: (1) a Sprite Generation Module ( ) that outputs K sprites S, (2) a Transformation Module ( ) that takes as input an image I and the sprites S to predict transformed sprites $\bar { S } ^ { I }$ , (3) a Decision Module ( ) that takes the image I and transformed sprites $\hat { S } ^ { \hat { I } }$ as input and outputs a probability distribution $p ^ { I }$ for using the sprites, and (4) a Training Criteria ( ) which consist of a reconstruction loss and potential regularization terms.这张图概括 Deep Sprite-based Image Models 的整体方法流程。阅读时先看模块之间传递的训练信号，再看作者如何把目标拆成可优化的子问题。
