Publication: Advantages of mutation in passive testing: An empirical study
Loading...
Files
Full text at PDC
Publication Date
2009-04
Advisors (or tutors)
Editors
Journal Title
Journal ISSN
Volume Title
Publisher
IEEE COMPUTER SOC
Citations
Exportar
Abstract
This paper presents an empirical study of the mutation techniques used by the tool PASTE. This tool allows the automation of our passive testing methodology for systems that present stochastic-time information. In our proposal, invariants express the fact that each time the implementation under test performs a given sequence of actions, then it must exhibit a behavior according to the probability distribution functions reflected in the invariant. We briefly review the theoretical framework of our methodology and the main features of our tool. Next, we present in detail the Mutants module that provides us with a functionality to test the effectiveness of invariants for detecting errors. Finally, we present a study of the results obtained from the performed experiments.
Description
International Conference on Software Testing, Verification, and Validation Workshops. APR 01-04, 2009. Denver, CO.
UCM subjects
Unesco subjects
Keywords
Citation
C. Andrés, M.G. Merayo, and M. Núñez. Passive testing of timed systems. In 6th Int. Symposium on Automated Technology for Verification and Analysis, ATVA’08, LNCS 5311, pages 418–427.Springer, 2008. An extended version is available at http://kimba.mat.ucm.es/manolo/papers/atva08-passive-extended.pdf.
C. Andrés, M.G. Merayo, and M. Núñez. Passive testing of stochastic timed systems. In 2nd Int. Conf. on Software Testing, Verification, and Validation, ICST’09 (in press). IEEE Computer Society Press, 2009.
J.H. Andrews, L.C. Briand, and Y. Labiche. Is mutation an appropriate tool for testing experiments? In 27th Int. Conf. on Software Engineering, ICSE’05, pages 402–411. ACM Press, 2005.
J.A. Arnedo, A. Cavalli, and M. Núñez. Fast testing of critical properties through passive testing. In 15th Int. Conf. on Testing Communicating Systems, Test-Com’03, LNCS 2644, pages 295–310. Springer, 2003.
E. Bayse, A. Cavalli, M. Núñez, and F. Zaïdi. A passive testing approach based on invariants: Application to theWAP. Computer Networks, 48(2):247–266, 2005.
B.S. Bosik and M.Ü. Uyar. Finite state machine based formal methods in protocol conformance testing. Computer Networks & ISDN Systems, 22:7–33, 1991.
L. Bottaci and E.S. Mresa. Efficiency of mutation operators and selectivemutation strategies: An empirical study. Software Testing, Verification and Reliability, 9(4):205–232, 1999.
D.A. Carrington and P.A: Stocks. A tale of two paradigms: Formal methods and software testing. In Z User Workshop, pages 51–68. Springer, Workshops in Computing, 1994.
A. Cavalli, C. Gervy, and S. Prokopenko. New approaches for passive testing using an extended finite state machine specification. Information and Software Technology, 45:837–852, 2003.
D. Clarke and I. Lee. Automatic generation of tests for timing constraints from requirements. In 3rd Workshop on Object-Oriented Real-Time Dependable Systems, WORDS’97, pages 199–206. IEEE Computer Society Press, 1997.
A. En-Nouaary, R. Dssouli, and F. Khendek. Timed Wp-method: Testing real time systems. IEEE Transactions on Software Engineering, 28(11):1024–1039, 2002.
S.C.P.F. Fabbri, J.C. Maldonado, T. Sugeta, and P.C. Masiero. Mutation testing applied to validate specifications based on statecharts. In 10th IEEE Int. Symposium on Software Reliability Engineering, ISSRE’99, pages 210–219. IEEE Computer Society Press, 1999.
R.M. Hierons and M.G. Merayo. Mutation testing from probabilistic finite state machines. In 3rd Workshop on Mutation Analysis, Mutation’07, pages 141– 150. IEEE Computer Society Press, 2007.
T. Higashino, A. Nakata, K. Taniguchi, and A. Cavalli. Generating test cases for a timed I/O automaton model. In 12th Int. Workshop on Testing of Communicating Systems, IWTCS’99, pages 197–214. Kluwer Academic Publishers, 1999.
W.E. Howden. Weak mutation testing and completeness of test sets.IEEE Transactions on Software Engineering, 8:371-379, 1982.
R. Lai. A survey of communication protocol testing. Journal of Systems and Software, 62:21–46, 2002.
D. Lee, D. Chen, R. Hao, R.E. Miller, J. Wu, and X. Yin. Network protocol system monitoring: a formal approach with passive testing. IEEE/ACM Transactions on Networking, 14:424–437, 2006.
D. Lee and M. Yannakakis. Principles and methods of testing finite state machines: A survey. Proceedings of the IEEE, 84(8):1090–1123, 1996.
D. Mandrioli, S. Morasca, and A. Morzenti. Generating test cases for real time systems from logic specifications. ACM Transactions on Computer Systems, 13(4):356–398, 1995.
M.G. Merayo, M. Núñez, and I. Rodríguez. Extending EFSMs to specify and test timed systems with action durations and timeouts. IEEE Transactions on Computers, 57(6):835–848, 2008.
R. Nilsson, J. Offutt, and J. Mellin. Test case generation for mutation-based testing of timeliness. In 2nd Workshop on Model Based Testing, MBT’06, pages 97–114. Electronic Notes in Theoretical Computer Science 164(4), 2006.
M. Núñez and I. Rodríguez. Conformance testing relations for timed systems. In 5th Int.Workshop on Formal Approaches to Software Testing, FATES’05, LNCS 3997, pages 103–117. Springer, 2006.
J. Offutt. A practical system formutation testing: Help for the common programmer. In 7th International Test Conference, ITC’94, pages 824–830. IEEE Computer Society Press, 1994.
I. Rodríguez, M.G. Merayo, and M. Núñez. HOTL: Hypotheses and observations testing logic. Journal of Logic and Algebraic Programming, 74(2):57–93, 2008.
J. Springintveld, F. Vaandrager, and P.R. D’Argenio. Testing timed automata. Theoretical Computer Science, 254(1-2):225–257, 2001. Previously appeared as Technical Report CTIT-97-17, University of Twente, 1997.
T. Sugeta, J.C. Maldonado, and W.E. Wong. Mutation testing applied to validate SDL specifications. In 16th IFIP Int. Conf. on Testing of Communicating Systems,TestCom’04, LNCS 2978, pages 193–208. Springer, 2004.
J. Tretmans. Testing concurrent systems: A formal approach. In 10th Int. Conf. on Concurrency Theory, CONCUR’99, LNCS 1664, pages 46–65. Springer, 1999.