Understanding Latent Vector Arithmetic for Attribute Manipulation in Normalizing Flows

Data Science master project by Eduard Gimenez Funes ’21

Five portraits of the same man with different facial expressions

Editor’s note: This post is part of a series showcasing Barcelona School of Economics master projects. The project is a required component of all BSE Master’s programs.

Abstract

Normalizing flows are an elegant approximation to generative modelling. It can be shown that learning a probability distribution of a continuous variable X is equivalent to learning a mapping f from the domain where X is defined to Rn is such that the final distribution is a Gaussian. In “Glow: Generative flow with invertible 1×1 convolutions,” Kingma et al introduced the Glow model. Normalizing flows arrange the latent space in such a way that feature additivity is possible, allowing synthetic image generation. For example, it is possible to take the image of a person not smiling, add a smile, and obtain the image of the same person smiling. Using the CelebA dataset we report new experimental properties of the latent space such as specular images and linear discrimination. Finally, we propose a mathematical framework that helps to understand why feature additivity works.

Conclusions

Generative Models for Deep Fake generation sit in between Engineering, Mathematics and Art. Trial and error is key to finding solutions to these types of problems. Theoretical grounding might only come afterwards. But when it does, it is simply amazing. By experimenting with normalizing flows we found properties of the latent space that have helped us create a mathematical model that explains why feature additivity works.

Connect with the author

About the BSE Master’s Program in Data Science Methodology