Publication: Modelo computacional de lectura cognitiva para la representación automática de textos
Loading...
Official URL
Full text at PDC
Publication Date
2008
Authors
Advisors (or tutors)
Editors
Journal Title
Journal ISSN
Volume Title
Publisher
Universidad Complutense de Madrid
Citations
Exportar
Abstract
El modelado del lenguaje natural en los ordenadores conlleva ciertas restricciones debido a la estructura lógica y a las limitaciones de tiempo y espacio de las máquinas, además de la complejidad intrínseca del lenguaje. Uno de los mayores problemas de dicho modelado es la representación de la semántica. Los primeros modelos conexionistas del lenguaje se situaban próximos a la cognición humana pero no eran lo suficientemente generales y eficientes para aplicaciones reales. Estos primeros sistemas de procesamiento de lenguaje natural hacían uso de redes de asociación como formalismo de representación. Debido a las limitaciones de almacenamiento y procesamiento de los ordenadores de aquella época, y al crecimiento de la información textual almacenada electrónicamente, los sistemas de procesamiento del lenguaje adoptaron formalismos matemáticos y estadísticos. Hoy en día, a causa de esa cantidad creciente de información textual los sistemas que son capaces de procesar textos son de extrema utilidad. Hasta hace r lativamente poco tiempo, la mayoría de estos sistemas utilizaban la clásica representación de los textos como “bolsa de palabras”, un formalismo de tipo vectorial que sólo tiene en cuenta las apariciones de las palabras de manera independiente. A mediados de los noventa, surgen los hiperespacios de palabras como un formalismo de representación alternativo al de “bolsa de palabras” tradicional. LSA (Análisis de Semántica Latente) fue el precursor de todos ellos, seguido por HAL (Hiperespacio Análogo al Lenguaje), PMI-IR, Indexado Aleatorio, WAS (Espacio de Asociación de Palabras) o ICAN (Construcción Incremental de una Red Asociativa), entre otros. Este tipo de sistemas construyen una representación en forma de matriz del conocimiento semántico lingüístico almacenado en una colección de textos dada. Este hiperespacio tiene en cuenta las relaciones entre las palabras y el contexto sintáctico y semántico en el que aparecen. Sin embargo, estos sistemas también representan los textos como vectores, llevando a cabo peraciones con las filas y las columnas de la matriz correspondientes a las palabras de los documentos. Aunque la representación mediante hiperespacios contiene mucha más información que la representación tradicional, puesto que los valores de los vectores son el resultado de la interacción entre las palabras y el contexto, los textos siguen siendo presentados como un conjunto de números sin estructura. A pesar de ello, los sistemas basados en hiperespacios han aportado una mejora significativa con respecto a los sistemas basados en la representación clásica. De los sistemas anteriormente mencionados, sólo ICAN introduce una representación estructural, almacenando el conocimiento en forma de red contextual asociativa de palabras y no como una matriz. Este modelo, a diferencia del resto de sistemas mencionados, hace posible la actualización del conocimiento sin necesidad de la reconstrucción total del mismo. A pesar del progreso realizado utilizando los hiperespacios de palabras, los seres humanos continúan r alizando tareas de procesamiento de lenguaje natural, como la clasificación de textos o la recuperación de información, de manera mucho más precisa que los ordenadores aunque, por supuesto, más despacio. Es difícil concebir el conocimiento lingüístico representado como una matriz en el cerebro humano, así como que la lectura suponga realizar operaciones matemáticas sobre dicha matriz. La lectura es un proceso secuencial de percepción en el tiempo, durante el cual los mecanismos mentales construyen imágenes e inferencias que se van reforzando, actualizando o descartando hasta la conclusión de la lectura del texto, momento en el que la imagen mental generada permite a los seres humanos resumir o clasificar el texto, recuperar documentos similares o simplemente expresar opiniones sobre el mismo. Esta es la filosofía que subyace en el sistema presentado en esta tesis. Este sistema, denominado SILC (Sistema de Indexación por Lectura Cognitiva), está ligeramente inspirado en el formalismo que sugiere el sistema ICA . Lo que se propone en este trabajo de tesis doctoral es un modelo computacional de lectura que construye una representación de la semántica de un texto como resultado de un proceso en el tiempo. Dicha representación posee una estructura que posibilita la descripción de las relaciones entre los conceptos leídos y su nivel de significación en cada momento del proceso de lectura. Existen otros modelos computacionales de lectura cuyo objetivo es más teórico que aplicado. La mayoría de ellos parten del modelo conexionista de Construcción-Integración y se centran en diferentes fases u objetivos de la lectura. Todos estos sistemas ponen de manifiesto la gran variedad y complejidad de los procesos cognitivos implicados en la lectura. El modelo propuesto en esta tesis, SILC, es un método sencillo que incluye sólo algunos de dichos procesos cognitivos y, aunque trata de ser útil en aplicaciones prácticas, está inspirado en los seres humanos tratando de asemejarse más a su proceder que el resto de sistemas del mismo ca po de aplicación. El modelo que implementa SILC intenta simular, en parte, procesos cognitivos de alto nivel que operan en el tiempo. Primero, el sistema construye una red de asociación conceptual como una memoria lingüística base a partir de una colección de textos que representan el espacio de conocimiento semántico. A continuación, el modelo genera representaciones de los textos de entrada como redes de conceptos con niveles de activación, que recogen el nivel de significación semántica de los mismos. Para ello, el modelo utiliza el conocimiento semántico lingüístico previamente construido realizando inferencias sobre el mismo mediante la propagación por la red de la activación de los conceptos leídos en orden secuencial. La representación generada se usa posteriormente para indexar documentos con el fin de clasificarlos automáticamente. Los métodos de indexación tradicionales representan los textos como resultado de procesos matemáticos. Puesto que los seres humanos superan ampliamente a los ordenadores e tareas de procesamiento de lenguaje natural, el modelo de SILC se inspira en la cognición humana para mejorar su eficacia en dichas tareas. Se han realizado experimentos para comparar el modelo con sujetos humanos, tanto durante la lectura, mediante la predicción o inferencia de conceptos, como al final de la misma, mediante la comparación con resúmenes generados por los sujetos. Los resultados muestran que el sistema es adecuado para modelar de manera aproximada el proceder humano en la lectura y sustentan la hipótesis de partida de SILC: cuanto más se asemeje el sistema a los seres humanos, mejor realizará las tareas prácticas del lenguaje. Los resultados también demuestran que el sistema es adecuado como marco experimental de validación de hipótesis relacionadas con aspectos cognitivos de la lectura. Otros experimentos de aplicación práctica han mostrado que, una vez que los parámetros del modelo han sido optimizados, la representación generada obtiene mejores resultados en clasificación de textos que otr representaciones generadas por los sistemas existentes. Se han definido tres medidas de similitud semántica entre textos a partir de las representaciones generadas por SILC. Los resultados experimentales muestran que la mejor de ellas es más eficaz y eficiente que otras medidas de similitud existentes. Además, la sinergia de dicha medida con el modelo de lectura implementado hace a SILC apropiado para su aplicación a tareas reales de procesamiento de lenguaje natural.
Modelling of natural language in computers implies some restrictions due to the logical structure and to the time and space limitations of machines, in addition to the language complexity itself. The computational representation of language semantics is one of the major problems. Connectionist models of language are close to human cognition but they are not general and efficient enough for real applications. The first natural language processing systems made use of association nets or graphs, generally speaking, as representation formalism. Due, in one hand, to the storage and processing limitations of the computers in that time and, in the other hand, to the need of processing a growing amount of textual information electronically stored, the language processing systems adopted mathematical and statistical formalisms. Nowadays, because of this huge amount of digital information stored in natural language, systems that automatically process text are of crucial importance and extremely useful. Until fairly rec ntly, most of the systems used the highly common electronic text representation, "bag of words". No information other than independent occurrences of words is considered in this latter vector-like formalism. In the mid-nineties, word hyperspaces were proposed as an alternative to the traditional "bag of words" approach. LSA (Latent Semantic Analysis) was the first of these systems, followed by HAL (Hyperspace Analogue to Language), PMI-IR, Random Indexing, WAS (Word Association Space) or ICAN (Incremental Construction of an Associative Network), among others. These kind of systems build a representation, a matrix, of the linguistic knowledge contained in a given text collection. The representation, or hyperspace, takes into account the relationship between words and the syntactic and semantic context where they occur, and this is the main difference with the common "bag of words" representation. However, once the hyperspace has been built, word hyperspace systems represent the text as a vector, and by doing o erations with the rows and the columns of the matrix corresponding to the words in the texts. Although the hyperspace representation contains much more information than the traditional representation because the vector values are the result of word and context interaction, texts are still a set of numbers without a structure. However, this approach has been shown to be a real improvement on the classical representation. Only ICAN introduces a structural representation and does not store linguistic knowledge as a matrix but as a net of associated words. This model makes it possible to incrementally add new words without retraining and recalculating the knowledge, which is psychologically more plausible. This approach proposes the representation of linguistic knowledge as a net of concepts associated by context. In spite of the progress made with word hyperspaces, human beings continue to do text classification and information retrieval tasks much better than machines, although of course more slowly. It is hard o believe that linguistic knowledge is represented as a matrix in the human mind and that reading implies mathematical operations on this matrix. Human reading is a process of sequential perception over time, during which the mind builds mental images and inferences which are reinforced, updated or discarded until the end of the text. At that moment, this mental image allows humans to summarize and classify the text, to retrieve similar texts or simply to talk about the text by expressing opinions. The latter dynamic is the one in which the system presented in this thesis relies. This system, called CRIM (Cognitive Reading Indexing Model), is inspired by the ICAN connectionist approach, where words and texts do not share the same structure of representation unlike the systems mentioned above. What is proposed in the PhD work is to build text representations as a result of a process over time, with a structure that makes it possible to indirectly describe the salience and relations of words at every instant d ring the reading process. Other computational models of reading exist which search for an assessment of a theory of reading rather than for a real data-intensive application. Most of them are based on connectionist networks inspired by the Construction-Integration model and they focus on different stages of reading and targets. These systems just mentioned show that there is a high number of complex cognitive processes underlying reading. The model proposed in this PhD thesis, so called CRIM, is a simple model that takes into account only a few cognitive processes and although it is aimed at a real application, it is inspired by and closer to human procedure than the other systems in the same application field. The CRIM tries to simulate in part the high-level cognitive processes in human mind over time. First, the system builds a conceptual association net from a collection of texts representing the semantic knowledge space, as a linguistic base memory. Then the model generates a representation of the input ext as a net of concepts, and each concept has an activation value referring to its salience in the text. This representation is then used to index documents in order to automatically categorize them by a supervised learning algorithm. Traditional indexing methods represent texts as the result of a process of mathematical operations. Since humans are able to classify texts much better than machines, the model is inspired in human cognition in order to improve language tasks. Some experiments were carried out to compare the model with humans, either during the reading process by concept prediction, or at the end of the reading process by summary comparison. The results showed that the system is suitable to model human reading process and proved the base hypothesis in which CRIM relies: the closer the system is to human being procedures, the better it performances in natural language processing tasks. Results also make the system suitable as an experimental framework to test hypothesis about other cognitive asp cts of reading. Other applied experiments show that, once the model parameters have been optimized, the representation obtained is an improvement on traditional indexing techniques. Given this representation, two different similarity measures between texts have been defined. The similarity measures are based in the distance between the single concepts of the texts and also on the difference of activation or significance. The distance between single concepts is calculated in the context defined by the compared texts and not in the global semantic net. The results show that the latter reduction improves both the efficiency and the accuracy of the comparison method, which is an improvement over other existent similarity measures between words. Other results of comparison between texts of different categories indicate that the synergy between the proposed model and similarity measures are very suitable to be applied in text categorization and information retrieval tasks.
Modelling of natural language in computers implies some restrictions due to the logical structure and to the time and space limitations of machines, in addition to the language complexity itself. The computational representation of language semantics is one of the major problems. Connectionist models of language are close to human cognition but they are not general and efficient enough for real applications. The first natural language processing systems made use of association nets or graphs, generally speaking, as representation formalism. Due, in one hand, to the storage and processing limitations of the computers in that time and, in the other hand, to the need of processing a growing amount of textual information electronically stored, the language processing systems adopted mathematical and statistical formalisms. Nowadays, because of this huge amount of digital information stored in natural language, systems that automatically process text are of crucial importance and extremely useful. Until fairly rec ntly, most of the systems used the highly common electronic text representation, "bag of words". No information other than independent occurrences of words is considered in this latter vector-like formalism. In the mid-nineties, word hyperspaces were proposed as an alternative to the traditional "bag of words" approach. LSA (Latent Semantic Analysis) was the first of these systems, followed by HAL (Hyperspace Analogue to Language), PMI-IR, Random Indexing, WAS (Word Association Space) or ICAN (Incremental Construction of an Associative Network), among others. These kind of systems build a representation, a matrix, of the linguistic knowledge contained in a given text collection. The representation, or hyperspace, takes into account the relationship between words and the syntactic and semantic context where they occur, and this is the main difference with the common "bag of words" representation. However, once the hyperspace has been built, word hyperspace systems represent the text as a vector, and by doing o erations with the rows and the columns of the matrix corresponding to the words in the texts. Although the hyperspace representation contains much more information than the traditional representation because the vector values are the result of word and context interaction, texts are still a set of numbers without a structure. However, this approach has been shown to be a real improvement on the classical representation. Only ICAN introduces a structural representation and does not store linguistic knowledge as a matrix but as a net of associated words. This model makes it possible to incrementally add new words without retraining and recalculating the knowledge, which is psychologically more plausible. This approach proposes the representation of linguistic knowledge as a net of concepts associated by context. In spite of the progress made with word hyperspaces, human beings continue to do text classification and information retrieval tasks much better than machines, although of course more slowly. It is hard o believe that linguistic knowledge is represented as a matrix in the human mind and that reading implies mathematical operations on this matrix. Human reading is a process of sequential perception over time, during which the mind builds mental images and inferences which are reinforced, updated or discarded until the end of the text. At that moment, this mental image allows humans to summarize and classify the text, to retrieve similar texts or simply to talk about the text by expressing opinions. The latter dynamic is the one in which the system presented in this thesis relies. This system, called CRIM (Cognitive Reading Indexing Model), is inspired by the ICAN connectionist approach, where words and texts do not share the same structure of representation unlike the systems mentioned above. What is proposed in the PhD work is to build text representations as a result of a process over time, with a structure that makes it possible to indirectly describe the salience and relations of words at every instant d ring the reading process. Other computational models of reading exist which search for an assessment of a theory of reading rather than for a real data-intensive application. Most of them are based on connectionist networks inspired by the Construction-Integration model and they focus on different stages of reading and targets. These systems just mentioned show that there is a high number of complex cognitive processes underlying reading. The model proposed in this PhD thesis, so called CRIM, is a simple model that takes into account only a few cognitive processes and although it is aimed at a real application, it is inspired by and closer to human procedure than the other systems in the same application field. The CRIM tries to simulate in part the high-level cognitive processes in human mind over time. First, the system builds a conceptual association net from a collection of texts representing the semantic knowledge space, as a linguistic base memory. Then the model generates a representation of the input ext as a net of concepts, and each concept has an activation value referring to its salience in the text. This representation is then used to index documents in order to automatically categorize them by a supervised learning algorithm. Traditional indexing methods represent texts as the result of a process of mathematical operations. Since humans are able to classify texts much better than machines, the model is inspired in human cognition in order to improve language tasks. Some experiments were carried out to compare the model with humans, either during the reading process by concept prediction, or at the end of the reading process by summary comparison. The results showed that the system is suitable to model human reading process and proved the base hypothesis in which CRIM relies: the closer the system is to human being procedures, the better it performances in natural language processing tasks. Results also make the system suitable as an experimental framework to test hypothesis about other cognitive asp cts of reading. Other applied experiments show that, once the model parameters have been optimized, the representation obtained is an improvement on traditional indexing techniques. Given this representation, two different similarity measures between texts have been defined. The similarity measures are based in the distance between the single concepts of the texts and also on the difference of activation or significance. The distance between single concepts is calculated in the context defined by the compared texts and not in the global semantic net. The results show that the latter reduction improves both the efficiency and the accuracy of the comparison method, which is an improvement over other existent similarity measures between words. Other results of comparison between texts of different categories indicate that the synergy between the proposed model and similarity measures are very suitable to be applied in text categorization and information retrieval tasks.
Description
Tesis de la Universidad Complutense de Madrid, Facultad de Informática, Departamento de Ingeniería del Software e Inteligencia Artificial, leída el 22-10-2007