Figureure 4 · : AVQ-Attention inference pipelineFig. 4: AVQ-Attention inference pipeline. Kernel 1 (VQ Precompute): keys and values are quantized against the parent codebook $C _ { p } ,$ , producing aggregated values $\bar { V } _ { p }$ and counts $n _ { p } .$ . Child quantization reuses parent assignments ○1 , so each key is compared only against the $\mathcal { C }$ children of its assigned parent. Kernel 2 (Flash Attention): Attn<sub>p</sub> computes attention over parent codewords and extracts importance. The online softmax accumulators and per-tile importance scores are carried forward $\textcircled{2}$ to Attn<sub>c</sub>, which refines the top ${ } _ { - } \mathcal { P }$ most important parents with child attention using correcting attention weights.这张图概括 AVQ-Attention 的整体方法流程。阅读时先看模块之间传递的训练信号，再看作者如何把目标拆成可优化的子问题。VQ codes per key (M or $M _ { 0 } { + } C )$ Attn codes per query (M or $M _ { 0 } + P C )VQ codes per key (M or $M _ { 0 } { + } C )$ Attn codes per query (M or $M _ { 0 } + P C )$ Fig. 6: Empirical verification of the complexity analysis in Tab. 1. Kernel time vs. effective codebook size on log-log axes for $\mathrm { V Q }$ -attention (blue) and AVQ-attention (red), at four sequence lengths $N$ (light to dark). Gray dashed lines indicate slope 1 (linear scaling). All kernels scale linearly with N (equidistant lines on the log scale for equal 4× increases in $N )$ and linearly with the number of codes, confirming $\mathcal { O } ( ( M _ { 0 } { + } C ) N D )$ for $\mathrm { V Q }$ precompute and $\mathcal { O } ( ( M _ { 0 } + \mathcal { P } \mathcal { C } ) N D )$ for attention. The precompute kernels appear slightly sub-linear in the number of codes; this is a fixed per-kernel overhead (launch cost, memory setup) that is amortized as the codebook grows.这张可视化用来解释 AVQ-Attention 学到的中间表征或对齐关系。重点看它是否支持正文里的机制判断。
