Metamorphic testing of OpenStreetMap



Downloads per month over past year

Almendros Jiménez, Jesús M. and Becerra Terón, Antonio and García Merayo, Mercedes and Núñez García, Manuel (2021) Metamorphic testing of OpenStreetMap. Information and Software Technology, 138 . p. 106631. ISSN 0950-5849

[thumbnail of 1-s2.0-S0950584921000987-main.pdf]
Creative Commons Attribution Non-commercial No Derivatives.


Official URL:


Context: OpenStreetMap represents a collaborative effort of many different and unrelated users to create a free map of the world. Although contributors follow some general guidelines, unsupervised additions are prone to include erroneous information. Unfortunately, it is impossible to automatically detect most of these issues because there does not exist an oracle to evaluate whether the information is correct or not. Metamorphic testing has shown to be very useful in assessing the correctness of very heterogeneous artifacts when oracles are not available.

Objective: The main goal of our work is to provide a (fully implemented) framework, based on metamorphic testing, that will support the analysis of the information provided in OpenStreetMap with the goal of detecting faulty information.

Method: We defined a general metamorphic testing framework to deal with OpenStreetMap. We identified a set of good metamorphic relations. In order to have as much automation as possible, we paid special attention to the automatic selection of follow-up inputs because they are fundamental to diminish manual testing. In order to assess the usefulness of our framework, we applied it to analyze maps of four cities in different continents. The rationale is that we would be dealing with different problems created by different contributors.

Results: We obtained experimental evidence that shows the potential value of our framework. The application of our framework to the analysis of the chosen cities revealed errors in all of them and in all the considered categories.

Conclusion: The experiments showed the usefulness of our framework to identify potential issues in the information appearing in OpenStreetMap. Although our metamorphic relations are very helpful, future users of the framework might identify other relations to deal with specific situations not covered by our relations. Since we provide a general pattern to define metamorphic relations, it is relatively easy to extend the existing framework. In particular, since all our metamorphic relations are implemented and the code is freely available, users have a pattern to implement new relations.

Item Type:Article
Additional Information:

CRUE-CSIC (Acuerdos Transformativos 2021)

Uncontrolled Keywords:Metamorphic testing, Quality of maps, OpenStreetMap
Subjects:Sciences > Computer science > Expert systems (Computer science)
Sciences > Computer science > Software
ID Code:70336
Deposited On:10 Feb 2022 13:26
Last Modified:18 Feb 2022 09:50

Origin of downloads

Repository Staff Only: item control page