Abstract: The interpretation of natural scenes, generally so obvious and effortless for humans, still remains a challenge in computer vision. To allow the search of image-based documents in digital libraries, we propose to design classifiers able to annotate images with keywords. First, we propose an image representation appropriate for scene description. Images are segmented into regions, and then indexed according to the presence of given region types. Second, we propound a classification scheme designed to separate images in the descriptor space. This is achieved by combining feature selection and kernel-method-based classification.