Automated testing in software development

The article considers one of the most important stages in the development of complex software systems - the testing phase. Modern development tools allow you to quickly build a "skeleton" application, but as far as quality? This article describes the main tasks of Software testing, types of tests and test criteria. Provides guidelines for the construction of the testing process.

Testing - one of the most important stages of checking the quality of developed software.

It so happens that the answer to the question about the quality of software in most cases gives only its operation - as the saying goes, "Life will show." However, the customer is entitled to require the developer guarantees. The abundance of unsuccessful attempts to get a custom high-quality software makes customers’ legal issues: whether it is possible in principle to create quality software? how? that should require the contractor to the customer to get quality software? should work as the customer himself?

To assess the quality of software has always used a combination of measures, including testing software to detect errors - one of the most important stages.

1. Where do you start testing? To conduct necessary testing facility - in this case software - and standard to which this object is compared. Software is a complex subject, which varies in composition and properties inspected at various stages of development. It is important to understand that if the developer and the customer does not have formulated requirements for the software before the start of its development, then, first, the customer is unlikely to get exactly what he wanted, and, secondly, the software that he did not receive be tested, because the object is, and there is no standard.

In other words, software testing is conducted in compliance with predetermined requirements (in terms of functionality, performance, security, etc.). Since the object of testing complex, requires a systematic approach to testing, its organization and conduct.

So the first conclusion that we need to clearly define requirements for software development at the beginning, but not "general requirements" such as "software should work," and claims that can be verified. The requirements for software are divided into functional (what functions and what should implement quality software) and non-functional (constraints on the time to solve the problem, the speed of data access requirements for the occupied resources, etc.). The customer and developer should be able to compare the current operation of the system, with its reference (expected) behavior. It is recommended to use an iterative approach, since the early testing of critical areas significantly reduces the risk of failure and the cost of corrections for the entire software development project.

2. Inexperienced customer is always set out to give the job and then see the final result. This is the surest way to get low-quality software. But the poor quality of disadvantageous to both parties. Because the customer ordered them not to use the software to the point where it is needed, and he loses time and money, because software should work on his business, but it does not work (ie, does not make money), and even selects the additional resources for improvement. The developer because he is losing credibility, is finalizing the software at their own expense (losing money) and can not quickly switch to another customer (lost orders).

 A much more effective in stages to control the progress of mining software. This approach brings the greatest effect and at a reasonable position the customer will likely be well received by the developer. For the success of the test is extremely important to carefully plan these activities. It is recommended to use document templates, including test plan, developed in accordance with the international standard IEEE 829-1983 Standard for Software Test Documentation.

We will not be in this article describe the entire process of software development. We only note that the discussion of technical specifications, technical design, system architecture with the customer - is also part of the process of detecting errors and clarify the standard. One of the main rules for the developer - should be very tight to work with the customer, then both sides will better understand what happens when you create software at any given time and how to quickly find joint solutions.

3. And yet, what does "phased testing"? We note first that many customers do not think that testing costs money and generally resource-intensive and that the quality you have to pay. However, realizing this, the customer should always be understood for what it pays and how to see the results.

Be divided by levels of testing tasks, and objects at different stages and phases of software development (see table):

testing software parts (modules, components) to verify the correct implementation of algorithms - is performed by developers; 

functional testing of subsystems and software as a whole in order to verify the extent to which functional requirements for the software - it is recommended to conduct a separate group of testers, not a subordinate leader development; 

stress testing (including stress) to identify the characteristics of functioning software load changes (intensity appeals to him, filling the database, etc.) - to do this work requires a highly expensive testers and automation of experiments.

When it is clear that and why should I test, and a plan of action, it’s time to think about how to do it better, faster and more accurately. Today’s software - is a complex object, and manually deal with them is difficult and expensive. Moreover, with "manual" test results of each test execution are lost and are difficult to repeat.

 In order to increase the volume of inspections and quality testing, to ensure the reuse of tests when making changes to software test automation tools are used. These include the automation of functional and load testing of client-server and Web-based applications: Rational TestStudio, Mercury LoadRunner, Segue SilkPerformer, as well as less popular products companies Compuware, CA, Empirix, RadView Software, etc.

Testing must be dealt with not only constant, but systematically. If you do not forget that this is the process of finding errors, it should require the developer to it periodically by specially created groups, a so-called "review", or "structural views" of the project materials and an audit of program source code. 

The customer has the right to negotiate with the developer of the commencement of such material or are not very deep monitoring the progress of that process. He is interested in the fact that the developer had put the organization processes. In this case, the customer can be assured that the quality of the developed software is controlled and ensured during development. That is what sent the well-known model of technological maturity of the CMM (Capability Maturity Model, www.sei.cmu.edu / cmmi / orgdocs / conops.html) and ISO 15 504.

It is desirable that at the stages of assembly, complex debugging and trial operation the developer fixed the error detection rate - then the variation of this intensity will be judged on the quality of software change, for example, whether to send it for trial or permanent operation.

 Finally, it is necessary to conduct complex test software for compliance with the TOR or other normative documents, the ability to work effectively with software based on the use of program documentation, Acceptance and other tests to ensure customer confidence in the performance created for his software.

And what about between trials, when the software is only created? And at this stage is very important to work on testing conducted according to plan. Although testing - the process is creative, it can and should be planned. This means that at each stage of work should be chosen quality criterion and the criterion of the test is complete. First need to tester or test team realized that to meet and what they check. 

That is what the object and the standard and some properties of the object scanned. The second criterion helps decide when the exhausted resources allocated to testing, he answers the question of under what circumstances testing can be completed.

The plan for testing is really needed. After checking for a complex object can and must use different strategies to achieve maximum results with existing restrictions on testing resources. For example, if you check the most likely path in the program, you can be sure that much to them that the software will work correctly. 

In addition, the test plan can determine in advance what to prepare for testing. For example, many software developers begin to prepare for the load experiments only when it is time for them. But it is known that such experiments need to test bench (a complex of equipment with the established system and application software), and you know what, you will need the original data base filled with data, its contents will change, and this base is sometimes you have to restore. 

Finally, you will need information, often referred to as normative or reference, and which should be stored in the software or databases.

The test plan is defined by international standard IEEE 829-1983. It should include at least three sections containing the following descriptions:

·         that will be tested (the test requirements, test cases);

·         and what methods will be tested in detail how the system;

·         schedule of work and resources required (staff, equipment).

Additionally describes the criteria for successful / unsuccessful completion of the tests, the testing criteria, risks, contingencies, provides links to relevant sections in the main documents of the project - plan requirements management, configuration management plan, etc.

How to prepare for testing, it was necessary to carry it out? As we have seen, you should check the requirements, then each of them associated with one or multiple test requirements. Next, a logical set of test requirements are grouped into test cases, test which will give a monosyllabic answer to a question, whether carried out correctly input is working correctly or that the business function in the system. This result is understandable not only to specialists, but also the end user. 

The main objects of the test automation - systems that implement the client side work. A key feature of the testing client-server systems is the ability to validate the operation and satisfactory performance of the system through the work of the client side. Thus, carefully and thoroughly checked these possibilities, we get a guarantee system performance for end users. 

Another important aspect of work organization - storage created tests. We recommend that you refer to the tests as well as to the source code, that is, to use versioned repository to test the possibility of recovery of previous versions (MS SourceSafe, Rational ClearCase ,…). It is useful at the stage of maintaining software and enable the reuse of ready-made tests on several versions of the system. Similarly, should apply to test data, creating files, copies and versions of database contents.

How to test? Testing - it’s always an experiment. To conduct necessary facilities. As in any experiment, the test need somewhere to collect the accumulated information, process the results. There is the largest division of types of testing: static and dynamic.

 In static testing the software is not executed, and analyzes the code, data structures. Dynamic testing, however, requires testing software. To do this requires not only test automation tools, and aids. We know very many tools for building and automatic generation of test tools for monitoring resources during test execution, measurement and visualization of test results of statistical processing of results, etc.

A limited amount of articles do not allow us to describe in detail the full range of activities to ensure customer confidence in the fact that software is well tested. In addition, there are always particular test-specific software. Not all developers have the necessary test automation tools. Many customers use for testing by independent organizations, charging them to outsource software testing. They do this for the sake of greater certainty as being created for them by, especially in these critical areas:

·         billing systems;

·         queuing system customers;

·         CRM-solutions;

·         Implementation of new ERP-system for assessing the quality of their work in a customer’s network and infrastructure in general;

·         testing software for its maintenance and development, supply of new versions and releases;

·         choice of servers that are running the software, etc.