Key Issues in Software Testing

Software testing - Activities performed to assess and improve software quality. This activity, in general, based on the detection of defects and problems in software systems. Testing of software systems consists of a dynamic verification of program behavior on a finite (limited) set of tests (set of test cases), selected as appropriate from the commonly performed operations applied field and provide verification of compliance with expected behavior of the system.

In accordance with IEEE Std 829-1983 Testing - a process analysis aimed at identifying differences between the real and the desired properties (defect) and to evaluate the properties of the software.

According to GOST R ISO IEC 12207-99 in the life cycle of software defined, among other supporting processes of verification, validation, joint review and audit.

The verification process is the process of determining what software products operate in full compliance with the requirements or conditions that are implemented in previous works. This process may include analysis, inspection and test.

The process of certification is the process of determining the completeness of compliance with established requirements established by the system or software product of their functional purpose.

Joint review process is a process of assessment and, where appropriate, results of products for the project.

The audit process is a process for determining compliance with the requirements, plans and contract terms.

In sum, these processes constitute what is commonly referred to testing.

Testing is based on the test procedures with specific input data, initial conditions and expected results developed for a specific purpose, such as checking a separate program or verification of compliance for a certain requirement. Test procedures can test different aspects of the program - a separate function to work properly to adequately fulfill the business requirements.

Testing is usually done in cycles, each of which has a specific list of goals and objectives. Testing cycle may coincide with or correspond to an iteration of a certain part. Typically, the cycle test is conducted for a particular build system.

Read More about Software testing services

Theoretical and practical limitations of testing

Theory Test against unreasonable level of confidence in the successful series of tests passed. Unfortunately, most of the established results of the theory test - negative, meaning, according to Dijkstra, that "the testing program can be used to demonstrate the presence of defects, but never show their absence." The main reason for this is that a comprehensive testing beyond the reach of real software.

Levels of testing

Testing is usually performed throughout the development and maintenance at various levels. The level of testing determines the "over what" made tests: on a separate module, a group of modules or system as a whole. However, none of the levels of testing can not be considered a priority. All levels of testing are important, regardless of the used models and methodologies.

·         Unit Testing. This level of testing makes it possible to test the individual elements of the system. What is considered an element - a module of the system is determined by context. The most complete form of this test is described in the standard IEEE 1008-87 «Standard for Software Unit Testing», giving the integrated concept of a systematic and documented approach to unit testing.

·         Integration Testing. This level of testing is the process of verification of the interaction between software components / modules.

·         System Testing. System testing covers their entire system. Most functional failures should be identified even at the level of unit and integration tests. In turn, system testing, typically focusing on non-functional requirements - security, performance, accuracy, reliability, etc. At this level, and tested interfaces to external applications, hardware, operating environment, etc.

Objectivies of Testing

Testing is conducted in accordance with specific goals (which may be explicitly or implicitly) and different levels of accuracy. Defining the purpose accurately, expressed quantitatively, enables control of test results. Test scenarios can be developed for testing the functional requirements (known as functional tests) and to evaluate non-functional requirements. In this case, there are tests, when quantified and test results can only speak indirectly to satisfy the objectives of testing (eg, «usability» - lightness, ease of use, in most cases can not be explicitly described quantitative characteristics).

There are the following, the most widespread and reasonable goals (and, accordingly, views) Test:

·         Acceptance / qualification testing. Examines the behavior of the system for customer satisfaction. This is possible if the customer takes the responsibility associated with such works as the party "host" software system, or routine tasks are specified, the successful validation (testing) which allows you to talk about customer satisfaction. Such tests can be conducted with the involvement of both system developers, and without them.

·         Installation testing. From the title implies that the tests are conducted to verify the installation procedure in the target system environment.

·         Alpha and beta testing. Before the software is available as a minimum, it must pass an alpha (internal test use) and beta (a test involving the use of selected external users) versions. Bug Reports received from users of these versions of the product processed in accordance with certain procedures, including confirmatory tests (any level) that are conducted by the team. This type of testing can not be pre-planned.

·         Conformance testing / Functional testing / Correctness testing. These tests may be called differently, but their essence is simple - check the system meets the requirements for her requirements described in the specification level of behavioral characteristics.

·         Reliability achievement and evaluation. Helping to identify the causes of failures, the testing means and improving the reliability of software systems. Randomly generated test cases can be used for statistical evaluation of reliability. Both goals - increasing reliability and evaluation - can be achieved by using models of reliability.

·         Regression testing. Determining the success of regression tests (IEEE 610-90 «Standard Glossary of Software Engineering Terminology») states: "The re-sample testing of the system or component to verify the modifications made should not lead to unintended effects." In practice this means that if the system has successfully passed tests before making any modifications, it should take them and after making such.

·         Performance Testing. Specialized tests for meeting the unique requirements of performance parameters. There is a separate subspecies of these tests, when an attempt is made to achieve the quantitative limits due to characteristics of the system and its operating environment.

·         Stress testing. Need to understand the difference between performance testing discussed above in order to achieve its real (achievable) possibilities of performance and execution of a software system increases the load c, up to the planned performance, and further enhanced behavior throughout the system load increase.

·         Back-to-back testing. A single set of tests to compare the two versions of the system.

·         Recovery testing. The purpose - to check the restart capabilities of the system in case of unforeseen disasters that affect the functioning of the operating environment in which to execute the system.

·         Configuration testing. In cases where software is created for use by various users, this type of testing aims to verify the behavior and performance of the system in various configurations.

·         Usability testing. Purpose - to test how easily the end user can master it, including not only the functional component - the system itself, but its documentation, how well you can perform tasks that automation is performed using this system, and finally, how well insured (with in terms of potential failures) of user errors.

·         Test-driven development. In fact, it is not so much the technique of testing, how much style organization of the design process life cycle, when the tests are an integral part of the requirements (and related specifications), rather than considered independent verification activities meet the requirements of the software system.