My current projects include:

Visual Question & Answering (VQA) for EO data

image-left

In Christel Chappuis’s PhD at EPFL/ECEO, co-supervised with Devis Tuia (EPFL) and Sylvain Lobry (Univ. of Paris), we investigate remote sensing visual question & answering (RSVQA). How to interact easily with Earth observation and geospatial data archives, using natural language and no computer expertise? This is key for empowering people with EO capacities! We explored image-text embedding for RSVQA (ECML-PKDD Workshop paper) and now are moving to advanced Natural Language Processing (NLP) techniques to address times-series of environmental data.

[ image-text embedding paper and video ]

Joint Energy-based Models for Generative EO Modelling

image-left

With Javiera Castillo-Navarro, Alex Boulch and Sebastien Lefevre we explored the potential of Energy-Based Models for generative modelling of Earth observation images. It leads to powerful applications such as image synthesis through Stochastic Gradient Langevin Dynamics, Out-Of-Distribution detection (see papers at ICLR/EBM ws and IGARSS in 2021), and Semi-Supervised Learning (see our TGRS paper).

[ EO-JEM preprint / EO-JEM in TGRS ]

Deep Interactive + Active Learning

image-left

In Gaston Lenczner’s PhD, we design interactive deep neural networks to foster user/algorithm collaboration. In the context of semantic segmentation of remote sensing images, we target several use-cases including online correction of semantic maps (and model!) obtained by CNNs, domain adaptation to new locations (see our ISPRS’2020 and MACLEAN 2021), and transfer learning to add interactively new target classes to an existing model (see our IGARSS2022 paper). To this end, we combine acceleration tricks and active learning to make deep networks learn continuously and efficiently from user inputs. This work is a collaboration with Guy Le Besnerais, Adrien Chan-Hon-Tong (both from ONERA) and Nicola Luminari (Alteia)

[ DISIR @ ISPRS 2020 / DISIR code / DISCA @ ECMLPKDD/MacLean 2021 (Best student paper award!) / DISCA Code / DIAL arxiv / Transfer learning arxiv ]


Older projects can be found here