We present a novel neural network architecture to estimate categorical emotions (e.g., happiness, sadness, anger…) and continuous emotions in terms of valence (how positive or negative the state of mind is) and arousal (how calming or exciting the experience is) with an unprecedented level of accuracy. In addition, our network is able to estimate facial landmarks at no additional cost and is suitable for real-time applications.

Training deep neural networks with spatio-temporal (i.e., 3D) or multidimensional convolutions of higher-order is computationally challenging due to the millions of unknown parameters across dozens of layers. In this paper, we propose a tensor factorization framework for efficient multidimensional (separable) convolutions of higher-order. Interestingly, the proposed framework enables a novel higher-order transduction, allowing to train a network on a given domain (e.g., 2D images or N-dimensional data in general) and using transduction to generalize to higher-order data such as videos (or (N+K)-dimensional data in general), capturing for instance temporal dynamics while preserving the learnt spatial information.

We present two novel and complimentary approaches to measure diffraction effects in commonly found planar spatially varying holographic surfaces. Such holographic surfaces are usually manufactured with one dimensional diffraction gratings that are varying in periodicity and orientation over an entire sample in order to produce a wide range of diffraction effects such as gradients and kinematic (rotational) effects. Our proposed methods estimate these two parameters and allow an accurate reproduction of these effects in real-time.

I have worked on the measurement and real-time rendering of diffraction effects during the first two years of my PhD. I am happy to say that the paper has been accepted at ACM Transactions On Graphics and that I will present it at SIGGRAPH 2017 in Los Angeles, California. The code for this project is available on my Github.

We propose a novel approach for real-time rendering of diffraction effects in surface reflectance in arbitrary environments using a low rank factorization of the diffraction kernel. We present realistic renderings in arbitrary environments and achieve a performance from 50 to 100 FPS making possible to use such a technique in real-time applications such as video games and VR.

We present a novel and practical approach for image-based relighting that employs the lights available in a regular room to acquire the reflectance field of an object. The lighting basis includes diverse light sources such as the house lights and the natural illumination coming from the windows. We achieve plausible results for diffuse and glossy objects that are qualitatively similar to results produced with dense sampling of the reflectance field including using a light stage and we demonstrate effective relighting results in two different room configurations. We believe our approach can be applied for practical relighting applications with general studio lighting.