iCLEF 2008


1) Search Task definition

The task is known-item image retrieval from Flickr. The user is given an image (not annotated) and the goal is to find the image again from Flickr using the system provided by iCLEF organizers. The user does not know in advance in which languages the image is annotated; therefore searching in multiple languages is essential to get optimal results.

The task is organized as an online game: the more images you find, the higher you will be ranked. In case of ties, the ranking will also depend on precision (number of images found / number of images attempted) and search time (less time is better). There will be a "Hall of Fame" with a rank of the best users. At any time whilst searching, the user will be allowed to quit the search (skip to next image) or ask for hints (a keyword or an indication on the target language).

2) Search interface

The iCLEF organizers provide a basic cross-language search front-end to Flickr, with the following basic functionalities:

NEW!! Flickling Interface available

For any questions or support using the interface, write to and you will normally receive a reply in around 24 hours.

3) Search logs

Search logs will be harvested from the search interface described above. Our intention is to have two main user groups:

  1. CLEF participants, who will be asked to interact with the system (the best team will receive an award at CLEF 2008). Besides contributing to the search logs, we believe that this experience will be an interesting hands-on experience for researchers working on CLIR that have never actually been users of a CLIR system.
  2. Flickr/web users at large. The game will be publicized in Flickr forums in order to increase the size of the search log.

4) What are iCLEF participants supposed to do?

iCLEF participants can essentially do two things:

iCLEF organizers will provide assistance to define the appropriate user groups, image lists, etc. within the common search interface. Besides these two options, and given the open spirit of iCLEF, we are open to groups having their own plans (for instance, their own interface designs) as long as they do not change the shared task(known-item search on Flickr).

5) OK, I'm interested. How do I join?

Please e-mail the stating your interest in the task, and go to the main CLEF page and follow the general registration procedure.

6) Schedule

Registration is now open. The only hard deadline is August 15, when the paper for the CLEF workshop notes is due. Meanwhile, iCLEF organizers will contact participants to assist them, promote discussions on the experimental design and retrieve information on the experiments for the iCLEF overview paper.

7) More Information

Read this workshop paper for more details about the task. Do not hesitate to for any question regarding iCLEF.