One ASCII file in XML format from each site. The format must conform this dtd . You can download an example of use .
The dtd imposes this format:
<iclef-results site="SITEID">
<experiment searcher="SEARCHERID" topic="QUESTIONNUM" system="SYSTEMID">
<doc docno="CLEFDOCNO" rel="RELEVANCEJUDGEMENT" time="TIME"/>
...
</experiment>
<experiment searcher="SEARCHERID" topic="QUESTIONNUM" system="SYSTEMID">
<doc docno="CLEFDOCNO" rel="RELEVANCEJUDGEMENT" time="TIME"/>
...
</experiment>
...
</iclef-results>
where:
SITEID : unique across sites
SYSTEMID : unique within site to each of your displaying systems.
SEARCHERID : unique within site to each of your searchers.
TOPICNUM : a CLEF-2002 topic number (one of 8 possible topic numbers)
CLEFDOCNO : a CLEF document identifier
RELEVANCEJUDGEMENT : one of the following values:
0 (not relevant)
1 (somewhat relevant)
2 (relevant)
3 (not judged)
(this field is pertinent only for comparing document selection strategies, see the general guidelines)
CONFIDENCE : one of the following:
0 (unsure)
1 (rather confident)
2 (certain)
(this field is pertinent only for comparing document selection strategies, see the general guidelines)
TIME (optional): elapsed time in seconds between document display and relevance judgment. A value of 0 should be provided when the judgment was made without examining the document or when the document was not judged.
The ranked list of documents retrieved after each interaction with the query translation/expansion interface, in exactly the same format as used for the standard CLEF runs. See the main CLEF guidelines for more details. The name of the run will be a concatenation of:
SITEID-SYSTEMID-SEARCHERID-INTERACTION
Where INTERACTION=0..n, where n is the number of times the user refines the original query.
Data must be submitted to flopez@lsi.uned.es as a zipped or gzipped file.
Fernando López Ostenero - Webmaster |
i CLEF ' 2005