久久影院一区二区三区-久久影院午夜伦手机不四虎卡-久久影院毛片一区二区-久久影视一区-在线精品91青草国产在线观看-在线激情小视频

<返回

Exploring Diffusion Time-steps for Unsupervised Representation Learning

Zhongqi Yue, Jiankun Wang, Qianru Sun, Lei Ji, Eric I-Chao Chang, Hanwang Zhang

ICLR 2024 Conference

May 2024

Keywords: unsupervised representation learning, diffusion model, representation disentanglement, counterfactual generation

Abstract:

Representation learning is all about discovering the hidden modular attributes that generate the data faithfully. We explore the potential of Denoising Diffusion Probabilistic Model (DM) in unsupervised learning of the modular attributes. We build a theoretical framework that connects the diffusion time-steps and the hidden attributes, which serves as an effective inductive bias for unsupervised learning. Specifically, the forward diffusion process incrementally adds Gaussian noise to samples at each time-step, which essentially collapses different samples into similar ones by losing attributes, e.g., fine-grained attributes such as texture are lost with less noise added (i.e., early time-steps), while coarse-grained ones such as shape are lost by adding more noise (i.e., late time-steps). To disentangle the modular attributes, at each time-step t, we learn a t-specific feature to compensate for the newly lost attribute, and the set of all {1,...,t}-specific features, corresponding to the cumulative set of lost attributes, are trained to make up for the reconstruction error of a pre-trained DM at time-step t. On CelebA, FFHQ, and Bedroom datasets, the learned feature significantly improves attribute classification and enables faithful counterfactual generation, e.g., interpolating only one specified attribute between two images, validating the disentanglement quality.

View More PDF>>

主站蜘蛛池模板: 泽州县| 容城县| 广灵县| 上饶市| 惠安县| 南木林县| 玛纳斯县| 凤冈县| 新乐市| 伊通| 清苑县| 沅陵县| 仙桃市| 时尚| 南澳县| 蒙城县| 垫江县| 花莲县| 萝北县| 永德县| 噶尔县| 黄石市| 浑源县| 当阳市| 孟津县| 永平县| 鄱阳县| 介休市| 尼木县| 虹口区| 齐河县| 从化市| 新竹县| 宜昌市| 沈丘县| 封丘县| 同心县| 朝阳区| 正宁县| 朝阳区| 铜梁县|