Unsupervised Classification and Learning-based Personalization

Talk at Inria Imedia research group.

Abstract: Due to the dramatic increase of the number of multimedia documents, content-based image retrieval has became an active research field, and applications such as digital libraries and video-on-demand emerged as real-life facts. If the feasibility of image retrieval using visual similarity has been proven, there are few attempts so far to give the user a way to deal globally with a large image collection.

We developed a clustering method able to find image categories in an image database. Such an automatic categorization requires to determine automatically the number of categories and to deal with challenging conditions since clusters are noisy, with various shapes and densities. Clustering is performed by minimizing a new Competitive Agglomeration objective function, in which competition is adaptive to the category densities. Beside, a noise cluster collects outliers, i.e. images which does not belong to a category, to prevent them to degrade clustering. By picking a prototype in each category, we are able to build a summary which could be used as a page zero for a CBIR system.

In a second step, we let the user organize the image collection. SVM learning is used to process feedback of category relevance. Only a few examples are required to filter and re-classify images to the user’s personally-defined categories. Training with such a small set is made possible by using the SVM kernel as a metric in the feature space and defining a distribution model for the data. This leads to the precise selection of relevant images.

Our method has been applied with success in an industrial context to macro-segmentation of video. Our system could generate a summary of broadcast news for fast Internet viewing. Another application is the clustering of similar image regions to allow image retrieval using query composition with multiple region prototypes.