Talk-to-Edit: Fine-Grained 2D and 3D Facial Editing via Dialog.

Facial editing is to manipulate the facial attributes of a given face image. Nowadays, with the development of generative models, users can easily generate 2D and 3D facial images with high fidelity and 3D-aware consistency. However, existing works are incapable of delivering a continuous and fine-grained editing mode (e.g., editing a slightly smiling face to a big laughing one) with natural interactions with users. In this work, we propose Talk-to-Edit, an interactive facial editing framework that performs fine-grained attribute manipulation through dialog between the user and the system. Our key insight is to model a continual "semantic field" in the GAN latent space. 1) Unlike previous works that regard the editing as traversing straight lines in the latent space, here the fine-grained editing is formulated as finding a curving trajectory that respects fine-grained attribute landscape on the semantic field. 2) The curvature at each step is location-specific and determined by the input image as well as the users' language requests. 3) To engage the users in a meaningful dialog, our system generates language feedback by considering both the user request and the current state of the semantic field. We demonstrate the effectiveness of our proposed framework on both 2D and 3D-aware generative models. We term the semantic field for the 3D-aware models as "tri-plane" flow, as it corresponds to the changes not only in the color space but also in the density space. We also contribute CelebA-Dialog, a visual-language facial editing dataset to facilitate large-scale study. Specifically, each image has manually annotated fine-grained attribute annotations as well as template-based textual descriptions in natural language. Extensive quantitative and qualitative experiments demonstrate the superiority of our framework in terms of 1) the smoothness of fine-grained editing, 2) the identity/attribute preservation, and 3) the visual photorealism and dialog fluency. Notably, the user study validates that our overall system is consistently favored by around 80% of the participants.

Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation.

Learning a good image prior is a long-term goal for image restoration and manipulation. While existing methods like deep image prior (DIP) capture low-level image statistics, there are still gaps toward an image prior that captures rich image semantics including color, spatial coherence, textures, and high-level concepts. This work presents an effective way to exploit the image prior captured by a generative adversarial network (GAN) trained on large-scale natural images. As shown in Fig. 1, the deep generative prior (DGP) provides compelling results to restore missing semantics, e.g., color, patch, resolution, of various degraded images. It also enables diverse image manipulation including random jittering, image morphing, and category transfer. Such highly flexible restoration and manipulation are made possible through relaxing the assumption of existing GAN inversion methods, which tend to fix the generator. Notably, we allow the generator to be fine-tuned on-the-fly in a progressive manner regularized by feature distance obtained by the discriminator in GAN. We show that these easy-to-implement and practical changes help preserve the reconstruction to remain in the manifold of nature images, and thus lead to more precise and faithful reconstruction for real images. Code is available at https://github.com/XingangPan/deep-generative-prior.

Time-dependent shrinkage of polymeric micelles of amphiphilic block copolymers containing semirigid oligocholate hydrophobes.

Alkyne-derivatized poly(ethylene glycol) (M.W. 5000) was coupled to several azide-terminated oligocholates by the click reaction to form amphiphilic block copolymers. A copolymer with a cholate hexamer as the hydrophobic block formed polymeric micelles that shrank by ~50% over a period of 10 h at 25°C. Shrinkage was faster and more dramatic at 35°C. Shortening the oligocholate by two units or inserting a 4-aminobutyroyl spacer in the hexacholate eliminated or diminished the shrinkage. Metastable aggregates were proposed to form when the block copolymers began to aggregate in water. The large hydrophobic surface, awkward shape, rigidity, and facial amphiphilicity of the cholate repeat unit and the long chain made it difficult for the oligocholates to adjust within the micellar core. As the oligocholates rearranged to maximize hydrophobic interactions and hydrogen-bonding while minimizing conformational strain, the polymeric micelles became more compact over time.

Efficient construction of oligocholate foldamers via "click" chemistry and their tolerance of structural heterogeneity.

The 1,3-dipolar cycloaddition between an alkynyl-terminated cholate trimer and an azido-functionalized cholate hexamer readily afforded the nonamer and dodecamer derivatives, whereas amide coupling employed in previous oligocholate synthesis failed beyond the octamer. Unlike typical oligocholate foldamers with exclusively head-to-tail arrangement of the repeat units, the newly synthesized "clicked" oligocholates contained head-to-head arrangement and flexible tethers in the sequence. Despite large structural perturbations, the clicked oligocholates folded similarly as the parent foldamers, demonstrating the robustness of the solvophobically driven folding mechanism.

Conformational interconversions in [2]catenanes containing a wide rigid bis(p-benzyl)methyl spacer.

The conformational interconversions of four [2]catenanes (1-4) containing a dibenzo-34-crown-10 ether (BPP34C10) interlocked with rings containing two 4,4'-dipyridiniums tethered by 1,3-bis(ethyloxy)phenyl and bis(p-benzyl)methyl spacers have been studied by VT 1H NMR spectroscopy. Symmetrically placed blocking groups on thickened tethers enabled either pathway for circumrotation of the BPP34C10 between isoenergetic sites to be blocked. On the basis of chemical shifts of the BPP34C10, its internal p-hydroquinone forms pi-pi-stacking interactions with only one 4,4'-dipyridinium ring at a time. The activation barrier for migration along either open tether was approximately 11.5 kcal/mol. This study demonstrates an ability to select the pathway for conformational interconversions in these [2]catenanes containing the rigid bis(p-benzyl)methyl tether and the lowering the barrier for interconversion through destabilization of the ground state structures.

Path selection for conformational interconversions in [2]catenanes.

[strucure: see text]The conformational interconversions of several [2]catenanes containing a dibenzo-34-crown-10 ether (BPP34C10) interlocked with rings containing two 4,4'-dipyridyls tethered by different aryl spacers have been studied. Blocking groups on the tethers enabled the two pathways for circumrotation of the BPP34C10 to be open or blocked. The activation barrier for migration along the open tethers varied from 11 to 13 kcal/mol. This study demonstrates an ability to select the pathway for conformational interconversions in [2]catenanes.