SummarySystems must emulate human assessment of QA responses and decide whether an Answer to a Question is correct or not according to a given Text.
Participant systems will receive a set of triplets (Question, Answer, Supporting Text) and they must return a boolean value for each triplet. Results will be evaluated against the QA human assessments. See Exercise Description for more details.
Keywords: Answer Validation, Question Answering, Recognising Textual Entailment (RTE)
Related Work: QA at CLEF, RTE Pascal Challenge
Promote the development and evaluation of subsystems aimed at validating the correctness of the answers given by a QA system. More information in
Anselmo Peñas, Álvaro Rodrigo, Valentín Sama, Felisa Verdejo. Testing the Reasoning for Question Answering Validation. Special Issue on Natural Language and Knowledge Representation, Journal of Logic and Computation. To appear ( Draft version )
|
|
|
