Publication: Mutation testing from probabilistic and stochastic finite state machines.
Loading...
Files
Full text at PDC
Publication Date
2009-11
Advisors (or tutors)
Editors
Journal Title
Journal ISSN
Volume Title
Publisher
Elsevier
Citations
Exportar
Abstract
Specification mutation involves mutating a specification, and for each mutation a test is derived that distinguishes the behaviours of the mutated and original specifications. This approach has been applied with finite state machine based models. This paper extends mutation testing to finite state machine models that contain non-functional roperties. The paper describes several ways of mutating a finite state machine with probabilities (PFSM) or stochastic time (PSFSM) attached to its transitions and shows how we can generate test sequences that distinguish between such a model and its mutants. Testing then involves applying each test sequence multiple times, observing the resultant behaviours and using results from statistical sampling theory in order to compare the observed frequency and execution time of each output sequence with that expected.
Description
UCM subjects
Unesco subjects
Keywords
Citation
Alur, R., Courcoubetis, C., Yannakakis, M., 1995. Distinguishing tests for nondeterministic and probabilistic machines. In: 27th ACM Symposium on Theory of Computing, STOC’95. ACM Press, pp. 363–372.
Baier, C., Kwiatkowska, M.Z., Norman, G., 1999. Computing probability bounds for linear time formulas over concurrent probabilistic systems. In: PROBMIV’98, Electronics Notes in Theoretical Computer Science, vol. 22.
Barnett, M., Grieskamp, W., Nachmanson, L., Schulte, W., Tillmann, N., Veanes, M., 2003. Towards a tool environment for model-based testing with AsmL. In: Third International Workshop on Formal Approaches to Testing of Software, FATES’03. LNCS, vol. 2931. Springer, pp. 252–266.
Bernardo, M., 2007. Non-bisimulation-based Markovian behavioral equivalences. Journal of Logic and Algebraic Programming 72 (1), 3–49.
Bernardo, M., Cleaveland, W.R., 2000. A theory of testing for markovian processes. In: 11th International Conference on Concurrency Theory, CONCUR’2000. LNCS, vol. 1877. Springer, pp. 305–319.
Bernardo, M., Gorrieri, R., 1998. A tutorial on EMPA: a theory of concurrent processes with nondeterminism, priorities, probabilities and time. Theoretical Computer Science 202 (1–2), 1–54.
Bernot, G., Bouaziz, L., LeGall, P., 1997. A theory of probabilistic functional testing. In: 19th IEEE International Conference on Software Engineering, ICSE’97. ACM Press, pp. 216–226.
Bottaci, L., Mresa, E.S., 1999. Efficiency of mutation operators and selective mutation strategies: an empirical study. Software Testing, Verification and Reliability 9 (4), 205–232.
Bravetti, M., Gorrieri, R., 2002. The theory of interactive generalized semi-Markov processes. Theoretical Computer Science 282 (1), 5–32.
Carrington, D.A., Stocks, P.A., 1994. A tale of two paradigms: formal methods and software testing. In: Z User Workshop. Workshops in Computing. Springer, pp. 51–68.
Cazorla, D., Cuartero, F., Valero, V., Pelayo, F.L., Pardo, J.J., 2003. Algebraic theory of probabilistic and non-deterministic processes. Journal of Logic and Algebraic Programming 55 (1–2), 57–103.
Chen, T.Y., Yu, Y.T., 1996. On the expected number of failures detected by subdomain testing and random testing. IEEE Transactions on Software Engineering 4 (2), 109–119.
Chow, T.S., 1978. Testing software design modelled by finite state machines. IEEE Transactions on Software Engineering 4, 178–187.
Clarke, D., Lee, I., 1997. Automatic generation of tests for timing constraints from requirements. In: Third Workshop on Object-Oriented Real-Time Dependable Systems, WORDS’97. IEEE Computer Society Press, pp. 199–206.
Cleaveland, R., Dayar, Z., Smolka, S.A., Yuen, S., 1999. Testing preorders for probabilistic processes. Information and Computation 154 (2), 93–148.
Couvreur, J.-M., Saheb, N., Sutre, G., 2003. An optimal automata approach to ltl model checking of probabilistic systems. In: 10th International Conference on Logic for Programming, Artificial Intelligence, and Reasoning, LPAR’03. LNCS, vol. 2850. Springer, pp. 361–375.
de Nicola, R., Hennessy, M.C.B., 1984. Testing equivalences for processes. Theoretical Computer Science 34, 83–133.
Duflot, M., Kwiatkowska, M.Z., Norman, G., Parker, D., 2006. A formal analysis of bluetooth device discovery. International Journal on Software Tools for Technology Transfer 8 (6), 621–632.
En-Nouaary, A., Dssouli, R., Khendek, F., 2002. Timed Wp-method: testing real time systems. IEEE Transactions on Software Engineering 28 (11), 1024–1039.
Fujiwara, S., Bochmann, G., Khendek, F., Amalou, M., Ghedamsi, A., 1991. Test selection based on finite-state models. IEEE Transactions on Software Engineering 17 (6), 591–603.
Gill, A., 1962. Introduction to The Theory of Finite State Machines. McGraw-Hill.
Hennessy, M., 1988. Algebraic Theory of Processes. MIT Press.
Hierons, R.M., 2004. Testing from a non-deterministic finite state machine using adaptive state counting. IEEE Transactions on Computers 53 (10), 1330–1342.
Hierons, R.M., Merayo, M.G., 2007. Mutation testing from probabilistic finite state machines. In: Third Workshop on Mutation Analysis, Mutation 2007. IEEE Computer Society Press, pp. 141–150.
Hierons, R.M., Ural, H., 2002. Reduced length checking sequences. IEEE Transactions on Computers 51 (9), 1111–1117.
Higashino, T., Nakata, A., Taniguchi, K., Cavalli, A., 1999. Generating test cases for a timed I/O automaton model. In: 12th International Workshop on Testing of Communicating Systems, IWTCS’99. Kluwer Academic Publishers., pp. 197–214.
Hillston, J., 1996. A Compositional Approach to Performance Modelling. Cambridge University Press.
Hoel, P.G., 1966. Elementary Statistics, second ed. John Wiley and Sons.
Howden, W.E., 1982. Weak mutation testing and completeness of test sets. IEEE Transactions on Software Engineering 8, 371–379.
Hwang, I., Kim, T., Hong, S., Lee, J., 2001. Test selection for a nondeterministic FSM. Computer Communications 24 (12), 1213–1223.
ITU-T, 1997. Recommendation Z.500 Framework on Formal Methods in Conformance Testing. International Telecommunications Union, Geneva, Switzerland.
Kwiatkowska, M., Norman, G., Parker, D., 2004. Probabilistic symbolic model checking with PRISM: a hybrid approach. International Journal of Software Tools for Technology Transfer 6 (2), 128–142.
Larsen, K., Skou, A., 1991. Bisimulation through probabilistic testing. Information and Computation 94 (1), 1–28.
Lee, D., Yannakakis, M., 1996. Principles and methods of testing finite state machines: a survey. Proceedings of the IEEE 84 (8), 1090–1123.
Lei, Y., Carver, R.H., Kacker, R., Kung, D., 2007. A combinatorial testing strategy for concurrent programs. Software Testing, Verification and Reliability 17 (4), 207– 225.
López, N., Núñez, M., 2001. A testing theory for generally distributed stochastic processes. In: 12th International Conference on Concurrency Theory, CONCUR’01. LNCS, vol. 2154. Springer, pp. 321–335.
Luo, G.L., von Bochmann, G., Petrenko, A., 1994. Test selection based on communicating nondeterministic finite-state machines using a generalized Wp-method. IEEE Transactions on Software Engineering 20 (2), 149–161.
Maldonado, J.C., Fabbri, S.C.P.F., Delamaro, M.E., Masiero, P.C., 1994. Mutation analysis testing for finite state machines. In: Fifth IEEE International Symposium on Software Reliability Engineering, ISSRE 1994. IEEE Computer Society Press, pp. 220–229.
Maldonado, J.C., Sugeta, T., Wong, W.E., 2004. Mutation testing applied to validate sdl specifications. In: 16th IFIP International Conference on Testing of Communicating Systems, TestCom 2004. LNCS, vol. 2978. Springer, pp. 193– 208.
Mandrioli, D., Morasca, S., Morzenti, A., 1995. Generating test cases for real time systems from logic specifications. ACM Transactions on Computer Systems 13 (4), 356–398.
Masiero, P.C., Delamaro, M.E., Fabbri, S.C.P.F., Maldonado, J.C., Wong, W.E., 1995. Mutation testing applied to validate specifications based on petri nets. In: Eighth International Conference on Formal Description Techniques, FORTE 95. Chapman & Hall, pp. 329–337.
Merayo, M.G., Núñez, M., 2007. Testing conformance on stochastic stream X-machines. In: 5th Software Engineering and Formal Methods, SEFM’07. IEEE Computer Society Press, pp. 227–236.
Merayo, M.G., Núñez, M., Rodríguez, I., 2007. Testing finite state machines presenting stochastic time and timeouts. In: 4th European Performance Engineering Workshop, EPEW’07. LNCS, vol. 4748. Springer, pp. 97–111.
Nandi, A., Marculescu, R., 2001. System-level power/performance analysis for embedded systems design. In: 38th ACM Design Automation Conference, DAC’01.
Nicollin, X., Sifakis, J., 1991. An overview and synthesis on timed process algebras. In: Third International Conference on Computer Aided Verification, CAV’91. LNCS, vol. 575, pp. 376–398.
Núñez, M., 2003. Algebraic theory of probabilistic processes. Journal of Logic and Algebraic Programming 56 (1–2), 117–177.
Núñez, M., Rodríguez, I., 2003. Towards testing stochastic timed systems. In: 23rd IFIP WG 6.1 International Conference on Formal Techniques for Networked and Distributed Systems, FORTE’03. LNCS, vol. 2767. Springer, pp. 335–350.
Petrenko, A., Yevtushenko, N., von Bochmann, G., 1996. Testing deterministic implementations from their nondeterministic FSM specifications. In: Ninth IFIP Workshop on Testing of ommunicating Systems, IWTCS’96. Chapman& Hall, pp. 125–140.
Reed, G.M., Roscoe, A.W., 1988. A timed model for communicating sequential processes. Theoretical Computer Science 58, 249–261.
Rouvellou, I., Hart, G.W., 1995. Inference of a probabilistic finite state machine from its output. IEEE Transactions on Systems, Manufacturing and Cybernetics 25 (3), 424–437.
Segala, R., 1996. Testing probabilistic automata. In: Seventh International Conference on Concurrency Theory, CONCUR’96. LNCS, vol. 1119. Springer, pp. 299–314.
Sen, K., Viswanathan, M., Agha, G., 2004. Statistical model checking of black-box probabilistic systems. In: 16th International Conference on Computer Aided Verification, CAV’04. LNCS, vol. 3114. Springer, pp. 202–215.
Springintveld, J., Vaandrager, F., D’Argenio, P.R., 2001. Testing timed automata. Theoretical Computer Science 254 (1–2), 225–257. Previously appeared as Technical Report CTIT-97-17, University of Twente, 1997.
Sugeta, T., Fabbri, S.C.P.F., Maldonado, J.C., Masiero, P.C., 1999. Mutation testing applied to validate specifications based on statecharts. In: 10th IEEE International Symposium on Software Reliability Engineering, ISSRE 1999. IEEE Computer Society Press, pp. 210–219.
Tzeng, W., 1992. A polynomial-time algorithm for the equivalence of probabilistic automata. SIAM Journal on Computing 21, 216–227.
van Glabbeek, R., Smolka, S.A., Steffen, B., 1995. Reactive, generative and stratified models of probabilistic processes. Information and Computation 121 (1), 59–80.
Vardi, M.Y., 1999. Probabilistic linear-time model checking: An overview of the automata-theoretic approach. In: Fifth International AMAST Workshop on Formal Methods for Real-Time and Probabilistic Systems, ARTS’99. LNCS, vol. 1601. Springer, pp. 265–276.
Vidal, E., Thollard, F., delaHiguera, C., Casacuberta, F., Carrasco, R., 2005. Probabilistic finite-state machines—Part 1. IEEE Transactions on Pattern Analysis and Machine Intelligence 27 (7), 1013–1025.
Wu, S.-H., Smolka, S.A., Stark, E.W., 1997. Composition and behaviors of probabilistic I/O automata. Theoretical Computer Science 176 (1-2), 1–37.
Zhang, F., Cheung, T.Y., 2003. Optimal transfer trees and distinguishing trees for testing observable nondeterministic finite-state machines. IEEE Transactions on Software Engineering 29 (1), 1–14.