My current projects include:
Visual Question & Answering (VQA) for EO data
In Christel Chappuis’s PhD at EPFL/ECEO, co-supervised with Devis Tuia (EPFL) and Sylvain Lobry (Univ. of Paris), we investigate remote sensing visual question & answering (RSVQA). How to interact easily with Earth observation and geospatial data archives, using natural language and no computer expertise? This is key for empowering people with EO capacities! We explored image-text embedding for RSVQA (ECML-PKDD Workshop paper) and now are moving to advanced Natural Language Processing (NLP) techniques to address times-series of environmental data.
Joint Energy-based Models for Generative EO Modelling
With Javiera Castillo-Navarro, Alex Boulch and Sebastien Lefevre we explored the potential of Energy-Based Models for generative modelling of Earth observation images. It leads to powerful applications such as image synthesis through Stochastic Gradient Langevin Dynamics, Out-Of-Distribution detection (see papers at ICLR/EBM ws and IGARSS in 2021), and Semi-Supervised Learning (see our TGRS paper).
Deep Interactive + Active Learning
In Gaston Lenczner’s PhD, we design interactive deep neural networks to foster user/algorithm collaboration. In the context of semantic segmentation of remote sensing images, we target several use-cases including online correction of semantic maps (and model!) obtained by CNNs, domain adaptation to new locations (see our ISPRS’2020 and MACLEAN 2021), and transfer learning to add interactively new target classes to an existing model (see our IGARSS2022 paper). To this end, we combine acceleration tricks and active learning to make deep networks learn continuously and efficiently from user inputs. This work is a collaboration with Guy Le Besnerais, Adrien Chan-Hon-Tong (both from ONERA) and Nicola Luminari (Alteia)
Older projects can be found here