Abstract
Case studies can help companies to evaluate the benefits of testing techniques and tools before their possible incorporation into the testing processes. Although general guidelines and organizational frameworks exist describing what a case study should consist of, no general methodological framework exists that can be instantiated to easily design case studies to evaluate different testing techniques. In this paper, we define the first version of a general methodological framework for evaluating software testing techniques, that focusses on the evaluation of effectiveness and efficiency. Using this framework, (1) software testing practitioners can more easily de_ne case studies through instantiation of the framework, (2) results can be better compared since they are all executed according to a similar design, and (3) the gap in existing work on methodological evaluation frameworks will be narrowed.
Keywords:
Case study, Software Testing Techniques, Methodological
Framework, Evaluation
Introduction
There exists a real need in the industry to have guidelines on what testing techniques to use for deterrent testing objectives, and how using these techniques are. In order to obtain such guidelines, more case studies need to be performed.
However, there is a lack of a general methodological evaluation framework that can simplify the design of case studies for comparing software testing techniques and make the results more precise, reliable, and easy to compare.
In this paper, we address this issue by presenting the first version of a general methodological framework for evaluating software testing techniques and tools1, which can be instantiated to define case studies that evaluate the effectiveness and efficiency of specific techniques and tools. The framework is general since it does not make any assumptions about the testing technique that is being evaluated nor about the subjects and the pilot projects. This framework has been systematically defined following the general guidelines to perform case studies defined by [11].
The remainder of the paper is organized as follows: Section 2 presents some relevant related work, Section 3 presents the general framework for the evaluation of testing techniques, and Section 4 presents our conclusions and future work.
Conclusions
In this paper, the first version of a methodological framework to evaluate software testing techniques has been presented. This framework will enable software testing practitioners to more easily define case studies by instantiating the framework while ensuring that the many guidelines and checklists for doing empirical work have been met. In addition, since case studies will be executed according to a similar design, it will be easier to compare the results obtained, and hence, specify general statements about testing techniques and tools evaluated.
As future work, we plan to improve the framework incorporating the evaluation of the subjective satisfaction of the users of testing techniques, and the feedback obtained by other researchers in national and international forums.
About KSRA
The Kavian Scientific Research Association (KSRA) is a non-profit research organization to provide research / educational services in December 2013. The members of the community had formed a virtual group on the Viber social network. The core of the Kavian Scientific Association was formed with these members as founders. These individuals, led by Professor Siavosh Kaviani, decided to launch a scientific / research association with an emphasis on education.
KSRA research association, as a non-profit research firm, is committed to providing research services in the field of knowledge. The main beneficiaries of this association are public or private knowledge-based companies, students, researchers, researchers, professors, universities, and industrial and semi-industrial centers around the world.
Our main services Based on Education for all Spectrum people in the world. We want to make an integration between researches and educations. We believe education is the main right of Human beings. So our services should be concentrated on inclusive education.
The KSRA team partners with local under-served communities around the world to improve the access to and quality of knowledge based on education, amplify and augment learning programs where they exist, and create new opportunities for e-learning where traditional education systems are lacking or non-existent.
FULL Paper PDF file
Evaluating Software Testing Techniques and ToolsAuthors
Tanja Vos, Beatriz Marn, Ignacio Panache, Arthur Baars, Claudia Ayala, and Xavier Franch
Acknowledgments
This work was funded by the MICINN project PROS-REQ (TIN2010-19130-C02-01, TIN2010-19130-C02-02) and the EU funded project FITTEST (ICT257574, 2010-2013).
Publisher
Universidade da Coruña
Citation
PDF reference and original file: Click here
Maryam kakaei was born in 1984 in Arak. She holds a Master's degree in Software Engineering from Azad University of Arak.
Professor Siavosh Kaviani was born in 1961 in Tehran. He had a professorship. He holds a Ph.D. in Software Engineering from the QL University of Software Development Methodology and an honorary Ph.D. from the University of Chelsea.
Nasim Gazerani was born in 1983 in Arak. She holds a Master's degree in Software Engineering from UM University of Malaysia.
