Understanding Latent Vector Arithmetic for Attribute Manipulation in Normalizing Flows

Data Science master project by Eduard Gimenez Funes ’21

Five portraits of the same man with different facial expressions

Editor’s note: This post is part of a series showcasing Barcelona School of Economics master projects. The project is a required component of all BSE Master’s programs.

Abstract

Normalizing flows are an elegant approximation to generative modelling. It can be shown that learning a probability distribution of a continuous variable X is equivalent to learning a mapping f from the domain where X is defined to Rn is such that the final distribution is a Gaussian. In “Glow: Generative flow with invertible 1×1 convolutions,” Kingma et al introduced the Glow model. Normalizing flows arrange the latent space in such a way that feature additivity is possible, allowing synthetic image generation. For example, it is possible to take the image of a person not smiling, add a smile, and obtain the image of the same person smiling. Using the CelebA dataset we report new experimental properties of the latent space such as specular images and linear discrimination. Finally, we propose a mathematical framework that helps to understand why feature additivity works.

Conclusions

Generative Models for Deep Fake generation sit in between Engineering, Mathematics and Art. Trial and error is key to finding solutions to these types of problems. Theoretical grounding might only come afterwards. But when it does, it is simply amazing. By experimenting with normalizing flows we found properties of the latent space that have helped us create a mathematical model that explains why feature additivity works.

Connect with the author

About the BSE Master’s Program in Data Science Methodology

BGSE Data Talks: Professor Piotr Zwiernik

The Barcelona GSE Data Science student blog has a new post featuring an interview with Piotr Zwiernik (UPF and BGSE), Data Science researcher and professor in the BGSE Data Science Master’s Program.

The Barcelona GSE Data Science student blog has a new post featuring an interview with Piotr Zwiernik (UPF and BGSE), Data Science researcher and professor in the BGSE Data Science Master’s Program:

Hello and welcome to the second edition of the „Data Talks“ segment of the Data Science student blog. Today we have the honor to interview Piotr Zwiernik, who is assistant professor at Universitat Pompeu Fabra. Professor Zwiernik was recently awarded the Beatriu de Pinós grant from the Catalan Agency for Management of University and Research Grants. In the Data Science Master’s Program he teaches the maths brush-up and the convex optimization part of the first term class „Deterministic Models and Optimization“. Furthermore, he is one of the leading researchers in the field of Gaussian Graphical Models and algebraic statistics. We discuss his personal path, the fascination for algebraic statistic as well as the epistemological question of low-dimensional structures in nature…

Read the full interview on the Barcelona GSE Data Scientists blog