Software Testing in Safety Critical Systems

Abstract Today, many arctic device- lively applications argon controlled by computer softw are. accordingly stiff foot raceing besidesls are essential to provide a high breaker point of condom and to reduce sedate disappointments too minimum. The paper examines existing regulating standards in golosh-critical arrangings. By comparing different parcel laddering methods the requirements and challenges in safety-critical software hearing are existence evaluated. The QUICKIES standard serves as the mall regulatory framework for all separately frames and provides the basis for the creation of application- and Inter thinkently tankards.Moreover it defines certain safety law trains depending on the cranial orbit of application and recommends runing methods consort to these levels. In model- based safety sorting a example model with restricted space pronounce domain is used to obtain representative test depicted objects. Statistical testing is a mathematical appeal that uses a high turning of test cases to drop dead a signifi pratt result. The main challenge of all safety- cerebrate testing methods Is to reduce testing cartridge clip and coordination compoundity without distorting the signifi tidy sumce of the test.These can for example be transportation systems, power plants, and medical applications. As peoples lives depend on the correct social occasion of such control systems and their software, thorough testing is required before they can be admitted to consummation. There are many different software testing methods. Most of them however poll the hazard of a failure but do not value its severity. However, in safety-critical systems a failure that has severe consequences, even if it is extremely rarified, can not be accepted. Therefore testing in this region has to be adopted accordingly.The purpose of this paper is to find and compare the latest methods for safety-critical footwear testing and to mark the most common i ndustry standard in this field. Moreover the requirements and challenges in safety-critical software testing impart be elaborated. At the beginning the paper allow for provide definitions that are required for the understanding of the concomitant chapters. After that, an introduction to the JUICE 508 safety standard, which serves as a basis for most industry-specific standards, is given.The chapter adjudicateing Methods will address some of the latest safety-related software testing methods in detail. 5 Definitions 2 Definitions 2. 1 Reliability and safe In safety critical systems both, dependability and safety are required to accomplish the goals of dependability. However, reliability and safety are two different attributes of dependability. The reliability, R(t) , of a system is a function of time. It is delimitate as the conditional hazard that the system will perform its think function in a defined way over a given time period and under certain stipulate and assumed c onditions.The most used parameter to characterize reliability is the Mean Time To Failure (MATT). The safety, S(t), of a system is defined as the probability that a system ill either perform its functions right on or will discontinue its functions in a way that does not interrupt the operation of opposite systems or Jeopardize the safety of any people associated with the system 1. Based on these definitions, in reliability testing all failures are weighted equally, whereas in safety testing the failures are weighted according to their severity.Therefore, a reliable system may be quite unsafe and a safe system may be very unreliable. 2. 2 preventive-critical System disk operating systems very complex to generate. As many states are unreachable or very difficult to reach hey can be reduced to a relatively small bit of representative system states. These states are grouped in tercet sub cross offs Normal State Subset (NUNS), Fail-Safe State subset (FPS) and Risky state subset ( IRS). Their relationships are s=Unusualness 6 Their inter-dependability is described as a Markova chain (see figure 1) 2. Figure 1 Three-state Markova Model for golosh-critical Systems(Source 2. Markova Chain Usage Model The Markova chain customs duty model describes the possible usage of a software based on a predicted environment. It can be used to generate statistical test cases and to estimate the software reliability. In an Markova model the transition from operation I to operation J can be denoted by an ordered pair . Let be the transition probability from operation I to operation J, with and EX=I .. N p(is)=1, where n is the number of trading operations. The transitions and transition probabilities can be represented in the form of a intercellular substance 3.Each specific usage of the program corresponds to a path X=(XI, XX, Xi) in the Markova chain where Xi corresponds to the I-the operation. P(Xi, X) determines the next executed operation J after execution of operation I. Since the operations are random rabbles, each path X=(XI, XX, ) forms a stochastic process. For a particular path x=(ox, XSL , ), the equal path execution probability is 3 7 pox pop , x 3 Standards There exist both national and outside(a) standards and guidelines at different depths and classifications which define requirements for safety-related technologies. Yester and provides the basis for the creation of application- and underspecified standards. It includes more than 500 pages of normative and informative specifications and proposals. Nowadays most safety-related standards are based on he JUICE 508 in cabal with the previously applicable requirements 4. The JUICE 508 defines so called Safety Integrity Levels (Sills) which serve as a measuring rod for the safety requirements on a certain system. The following table shows the different SILLS as well as the corresponding probability of failure and application examples.Probability of Failure One Failure in x eld Consequen ces Application Example The last three parts, are informative and include practical examples which should help to modify the application of the standard. The CE 61 508 describes the complete life cycle of safety-related systems from planning to decommissioning and refers to all aspects related to the use and requirements for electrical / electronic / programmable electronic systems (E / E / PEE) for separately functions 4. According to the focus of this paper lonesome(prenominal) the parts relating to software testing are mentioned in the following paragraph. Figure 2 shows the verification and validation process in software development according to the JUICE 508 standard.The E/E/PEE system safety requirements are applied both on the system architecture and the software specifications. Every level in the system architecture verifies if it meets the requirements of the next higher layer (I. E. Coding fulfills faculty design requirements, module design fulfills software yester des ign requirements etc. ). Moreover each system architecture layer is well-tried by a specific test. As soon as the test circuit is closed successfully the software can be validated. The standard also recommends and rates certain test methods according to the required SILL. In order to meet the requirements of the CE standard a series.Test methods comprised in the CE 61 508 are categorized as follows 6 Failure analysis (I. E. Cause consequence programs) Dynamic analysis and testing (I. E. Test case execution from model-based test case generation) Functional and black box testing (I. . Equivalence classes and arousal partition testing, including boundary value analysis) Performance testing (I. E. Response timings and memory constraints) unchanging analysis (I. E. Static analysis of run time error behavior) 9 Figure 2 CE 61 508-3 cheque and Validation Process(Source 10 scrutiny Methods 4 Testing Methods There are many different software testing methods.A detailed introduction to al l different methods would be far beyond the scope of this paper. Therefore the author will only mention two methods he deems most relevant in the field of safety-related software testing. Finally both methods are compared and their possible application areas are evaluated. 4. 1 Model-based Safety Testing In model-based testing explicit behavior models that encode the intended behavior of a system and its environment are used. These models generate pairs of inputs and outputs. The output of such a model represents the pass judgment output of the system under test (SOT). Mineral model-based testing method. The system safety-related behavior is defined in the safety requirements specification. Test cases are derived from a safety model that is extracted from the shut and from chunk safety requirements. This model encodes the intended behavior and maps each possible input to the corresponding output. Safety test selection criteria relate to the useable safety of the safety- critical system, to the structure of the model (state coverage, transition coverage), and also to a well defined set of system faults.Safety test case specifications are used to formalize the safety test selection criteria and turn back them operational. For the given safety model and the safety test case specification, an automatic safety test case generator and optimizer generates the safety test case suite. Finally, the concreted input part of a test case is submitted to the SHUT and the SOTs output is recorded. The concentration of the input part of a test case is performed by a safety test engine. Besides executing the safety case, it can also compare the output of the SHUT with the expected output as provided by the safety test case 6. 1 Figure 3 Model-based Safety Testing according Gang You et al. (Source Test Case Generation One of the most commonly tools for test case generation are model checking techniques. The main purpose of model checking is to verify a formal safety property (given as a logic formula) on a system model. In test case generation, model checking is used in order to find violations of certain formal safety properties. Safety models of safety-critical software systems may have a huge number of states. Therefore the greatest challenge when using a model checker is to cope with the state space explosion.As a countermeasure, Gang You et al. s approach applies the safety model, which is derived from SHUT and certain safety requirements. The model 12 limits the number of states by splitting them into three subsets (NUNS, FPS, IRS) containing only representative states (see 2. X). Moreover the safety model encodes he intended behavior, and from its structure, safety test cases can be derived. It thereby restricts the possible inputs into the SHUT and the set of possible separately behaviors of the SOT.Hence, to reduce the amount of testing and guarantee the quality of testing the model checker will search those most frequently entered states and generate the corresponding safety test cases without searching the whole state spaces. The selection of states is based on the safety requirements (Sills). Generally speaking, the safety model can be seen as a test selection mensuration generate safety-related test cases. Figure 4 shows the corresponding flow chart. 1 . The system safety model in the form of a finite state machine (FSML) is transformed into the input diction of the model checker tool (SPIN) 2.Each test requirement of a given safety criterion is develop as a temporal logic expression (LET). 3. Based on the Markova model of a system, the state space is divided into three subsets. 4. In term of these subsets, the negation of each expression of the formula is verified by the model checker. If there is an execution path in the model that does not satisfy the negated formula then it is presented by the model checker as a counter-example. This path becomes a test sequence that satisfies the sea captain test requiremen t. 5.The inputs and outputs that form the executable test case are extracted from the counter-example or are derived by a corresponding guided simulation of the model. 13 Figure 4 Test Case Generation fashion model according Gang You et al. (Source 4. 2 Statistical Testing As already mentioned in 2. 1 reliability is defined as the conditional probability that the system will perform its intended function. This chapter will splice the reliability of a system with the Markova usage model (see 2. 3). Let f be a function that shows the failure probability of a software. The argument D represents the possible usage set of the software.Each element AXED is a usage path from quo (initial operation) to send (final operation) The relation between software reliability R and failure probability F is R=l -F (2). In the assumed model the failure behavior of the software only depends on its usage path X and not on the input. This misbegottens that the input domain corresponding to the used X i s homogeneous. The simplest way of obtaining unbiased reliability estimation of the software is to select N test paths XSL, XX, , CNN according to the usage model. The exult of the function f(Xi) is 1 if the path fails and O otherwise.Then the arithmetic 14 mean of f(Xi) is an unbiased estimate PEP(f(X)), which is the mathematical expectation of the software failure probability under transition matrix P. Hence, the software reliability can be expressed as R=l -PEP(f(X)) 3. Critical operations are infrequently executed in real applications. This generates the problem that development organizations have to spend too much time when performing adequate statistical testing. Although one can overcome these drawbacks by increase the execution probabilities of critical operations during statistical entire software under test. Yang Going et al. 3 found a possible approach to overcome this problem Importance Sampling (IS) Based Safety-critical Software Statistical Testing Acceleration. IS Ba sed Safety-critical Software Statistical Testing Acceleration This chapter presents the Is-based software statistical testing acceleration method. It ensures that the critical operations tested adequately by adjusting the transition probabilities in the matrix of the usage model, and at the analogous time, produces the unbiased reliability of the software under test. The IS technique reduces simulation run times hen estimating the probabilities of rare events by Monte Carlo simulations 3.For complex software with a large model matrix, the simulation procedure is oft extremely time consuming. To overcome this problem, Yang Going et al. s approach adopts a simulated annealing algorithmic rule to calculate the optimal matrix Q. This widely used optimization method employs stochastic techniques to avoid being trapped in local optimal solution. The 16 exact mathematical explanation of this algorithm is complex and would be out of the scope of this paper. 3 4. 3 Method Comparison Altho ugh model-based and statistical testing follow exclusively different approaches, the challenges are very similar.Both methods have to limit the extent and complexity of testing. Model-based testing reduces the number of test cases by restricting the state space domain of the Markova chain usage model. Whereas statistical testing reduces the number by changing the relation between critical and normal test cases with help off likelihood ratio. 5 Conclusion Today an increasing number of safety-critical applications are controlled by computer software. Therefore effective testing tools are required to provide a high degree of safety and to reduce severe failures to a minimum. The paper focused on