Biblioteca de la Universidad Complutense de Madrid

Question answering pilot task at CLEF 2004

Impacto



Herrera, Jesús y Peñas, Anselmo y Verdejo, Felisa (2005) Question answering pilot task at CLEF 2004. In Multilingual Information Access for Text, Speech and Images: 5th Workshop of the Cross-Language Evaluation Forum, CLEF 2004, Bath, UK. Lecture Notes in Computer Science, 3491 . Springer-Verlag, Berlin, pp. 581-590. ISBN 3 540 27420 0

[img]
Vista previa
PDF
141kB

URL Oficial: http://nlp.uned.es/pergamus/pubs/QApilotTaskatCLEF2004.pdf



Resumen

A Pilot Question Answering Task has been activated in the Cross-Language Evaluation Forum 2004 with a twofold objective. In the first place, the evaluation of Question Answering systems when they have to answer conjunctive lists, disjunctive lists and questions with temporal restrictions. In the second place, the evaluation of systems’ capability to give an accurate self-scoring about the confidence on their answers. In this way, two measures have been designed to be applied on all these different types of questions and to reward systems that give a confidence score with a high correlation with the human assessments. The forty eight runs submitted to the Question Answering Main Track have been taken as a case of study, confirming that some systems are able to give a very accurate score and showing how the measures proposed reward this fact.


Tipo de documento:Sección de libro
Palabras clave:Pregunta respuesta; Evaluación; Medida de evaluación; Lista conjuntiva; Lista disyuntiva; Restricción temporal; Autoevaluación de la confianza; Question answering; Evaluation; Evaluation measure; Conjunctive list; Disjunctive list; Temporal restriction; Confidence self-scoring
Materias:Ciencias > Informática > Inteligencia artificial
Código ID:5529
Depositado:03 Abr 2006
Última Modificación:06 Feb 2014 07:44

Sólo personal del repositorio: página de control del artículo