The methods of designing test cases with black-box technology include equivalence class division, boundary value analysis, wrong speculation, causality diagram and comprehensive strategy.
Black-box testing focuses on testing the functional requirements of software, that is, black-box testing enables software engineers to derive the input conditions for executing all functional requirements of the program. Black-box testing is not a substitute for white-box testing, but is used to help white-box testing find other types of errors.
Black box testing attempts to find the following types of errors:
1) function error or omission;
2) Interface error;
3) Data structure or external database access error;
4) performance error;
5) Initialization and termination errors.
First, the test case design method of black-box testing
Method of dividing equivalence classes
boundary value analysis
Wrong speculation method
Causality diagram method
Decision table-driven analysis method
Orthogonal experimental design method
Functional diagram analysis method
Equivalence class division:
It is to divide all possible input data, that is, the input field of a program into several parts (subsets), and then select several representative data from each subset as test cases. This method is an important and commonly used method to design black-box test cases.
1) Divide equivalence class: equivalence class refers to a subset of an input field. In this subset, all the input data is equivalent to exposing errors in the program. It is reasonable to assume that testing the representative value of an equivalent class is equivalent to testing other values of this class. Therefore, all the input data can be reasonably divided into several equivalence classes, and if one data in each equivalence class is taken as the input condition of the test, a small amount of representativeness can be used.
Effective equivalence class: refers to a group of input data that is reasonable and meaningful to the specification of the program. Using effective equivalence classes, we can check whether the program has achieved the functions and performance specified in the specification.
Invalid equivalence class: the definition of valid equivalence class is just the opposite.
When designing test cases, we should consider these two equivalence classes at the same time, because software should not only be able to receive reasonable data, but also be able to withstand unexpected tests. Such a test can ensure that the software has higher reliability.
2) Method of dividing equivalence classes: Here are six principles for determining equivalence classes.
① Under the condition that the input conditions specify the range or number of values, one valid equivalence class and two invalid equivalence classes can be established.
② Under the condition that the input conditions specify the set of input values or "what must be done", valid equivalence classes and invalid equivalence classes can be established.
(3) If the input condition is a Boolean quantity, the valid equivalence class and the invalid equivalence class can be determined.
(4) If a set of values of input data is specified (assuming n) and the program must deal with each input value separately, n valid equivalence classes and one invalid equivalence class can be established.
⑤ Under the condition that the input data must abide by the rules, one valid equivalence class (conforming to the rules) and several invalid equivalence classes (violating the rules from different angles) can be established.
⑥ If it is known that each element in the divided equivalence class has different ways in program processing, then the equivalence class should be further divided into smaller equivalence classes.
3) Design test cases: After the equivalence classes are established, an equivalence class table can be established to list all the divided equivalence classes:
Input conditional valid equivalence class invalid equivalence class
.........
.........
Then design test cases from the divided equivalent classes according to the following three principles:
① Assign a unique number to each equivalence class.
② Design a new test case, covering as many effective equivalence classes as possible, and repeat this step until all effective equivalence classes are covered.
(3) Design a new test case, covering only one uncovered invalid equivalent class, and repeat this step until all the invalid equivalent classes are covered.
Boundary value analysis
The boundary value analysis method is a supplement to the equivalence class division method.
Thinking of (1) boundary value analysis method;
Long-term test experience tells us that a large number of errors occur at the boundary of the input or output range, rather than within the input and output range. Therefore, by designing test cases for various boundary conditions, more errors can be found.
When designing test cases with boundary value analysis, the boundary conditions must be determined first. Usually, the boundary of input-output equivalence class is the boundary condition that should be tested emphatically. You should choose a value just equal to, slightly larger than or slightly smaller than the boundary as the test data, rather than any value in the typical value or equivalent class.
(2) Test case selection principles based on boundary value analysis:
1) If the input condition specifies a range of values, the values just reaching the boundary of this range and the values just beyond this range should be used as test input data.
2) If the input condition specifies the number of values, the maximum number, the minimum number, the number less than the minimum number and the number greater than the maximum number are used as test data.
3) According to each output condition specified in the specification, use the previous principle 1).
4) Apply the previous principle 2) according to each output condition specified in the specification.
5) If the input field or output field given in the program specification is an ordered set, the first element and the last element of the set should be selected as test cases.
6) If an internal data structure is used in the program, the value on the boundary of the internal data structure should be selected as the test case.
7) Analyze the specification and find out other possible boundary conditions.
False conjecture method
Error inference method: A method of designing test cases according to experience and intuition and inferring all possible errors in the program.
The basic idea of error reasoning method: list all possible errors and special cases that are prone to errors in the program, and select test cases accordingly. For example, many common mistakes in modules listed in unit tests, mistakes found in previous product tests, and so on. These are lessons learned. In addition, when the input data and output data are 0, the input table is blank or the input table has only one row. These are all error-prone situations. Examples of these situations can be selected as test cases.
Causality diagram method
The above-mentioned equivalence class division method and boundary value analysis method all focus on input conditions, without considering the relationship and combination of input conditions. Considering the mutual combination of input conditions, some new situations may appear. However, it is not easy to check the combination of input conditions. Even if all input conditions are divided into equivalence classes, there are quite a few combinations between them. Therefore, we must consider adopting a form that is suitable for describing the combination of multiple conditions and generating multiple actions accordingly.
The final result of causality chart is a decision table, which is suitable for checking various combinations of program input conditions.
Basic steps of generating test cases by using causality diagram:
(1) Analyze the descriptions of software specifications, which are causes (i.e. input conditions or equivalent classes of input conditions) and results (i.e. output conditions), and assign an identifier to each cause and result.
(2) Analyze the semantics in the description of software specification, find out the corresponding relationship between cause and result, and draw a causal diagram according to these relationships.
(3) Due to the limitation of grammar or environment, some combinations between cause and effect are not impossible. In order to express these special situations, some symbols are used to express constraints or restrictive conditions on the causality diagram.
(4) Convert the causality diagram into a decision table.
(5) Take out each column of the judgment table as the basis for designing test cases.
The test cases generated by causality diagram (under partial and combined relations) contain all the true and false input data, and the number of test cases is the least. With the increase of input data, the number of test cases increases linearly.
Decision table is used in causality diagram method. Decision table is a tool to analyze and express different operations under multi-logic conditions. In the early development of programming, decision table has been used as an auxiliary tool for programming, because it can express complex logical relations and combinations of various conditions concretely and clearly.
The decision table usually consists of four parts.
Condition Stub: Lists all conditions of the problem. It is generally considered that the order of the listed conditions is not important.
Action Stub: Lists the possible actions specified by the question. The order of these actions is not limited.
Condition entry: lists the value of the condition in its left column, as well as all possible true and false values.
Action items: list the actions that should be taken under various conditions.
Rule: the specific value of any combination of conditions and the corresponding action to be performed. The column that runs through the condition item and the action item in the decision table is the rule. Obviously, as many groups of condition values are listed in the decision table, there are many rules, that is, there are many columns of condition items and operation items.
Steps of establishing decision table: (according to software specification)
① Determine the number of rules. If there are n conditions, each condition has two values (0, 1), so there are rules.
② List all condition piles and action piles.
③ Fill in the conditions.
④ Fill in the action items. Wait until the preliminary judgment table.
⑤ Simplify merging similarity rules (same action).
B. Baesel pointed out the conditions suitable for designing test cases using decision tables:
Specifications are given in the form of decision tables, which can also be easily converted into decision tables.
The order of conditions does not and will not affect which operations are performed.
③ The order of rules will not and will not affect which operations are performed.
(4) Every time the condition of a rule has been met and the operation to be performed has been determined, there is no need to check other rules.
⑤ If a rule is met, multiple operations will be executed, and the execution order of these operations is irrelevant.
Advantages of black box testing
1. Basically nobody cares. If the program stops running, it is usually the program under test that crashes.
2. After designing the test case, the next work is cool, and of course, what is more depressing is to determine the cause of the crash.
Disadvantages of black box testing
1. The result depends on the design of the test case. The design of test cases comes from experience, and OUSPG is worth learning from.
2. There is no concept of state transition. At present, there are basically some successful examples for PDU, but it is impossible to make state transition for the tested program.
3. For the test without the concept of state, it is troublesome to find and determine the test cases that cause the program to crash, and the test cases that may appear around must be confirmed separately. As far as state testing is concerned, it is even more troublesome, especially if it is not caused by a single test case. These are more prominent on the issue of heap.
Selection of black box testing (functional testing) tools
So, how to complete the functional test efficiently? It is undoubtedly very important to choose a suitable functional testing tool and cultivate a high-quality tool user team. Although a few software service enterprises are engaged in functional testing outsourcing projects without using any functional testing tools at this stage. In the short run, the profitability of such enterprises is acceptable, but in the long run, it is likely to be replaced by software service enterprises with higher automation.
At present, there are many software tools for functional testing, and tools for different architectures are constantly being introduced. This paper focuses on a typical automated testing tool, namely WinRunner of Mercury Company.
WinRunner is an enterprise-level software function testing tool to test whether the application can run as scheduled. By automatically capturing, detecting and simulating user interaction, WinRunner can identify most software functional defects, thus ensuring that applications that span multiple functional points and databases will not have functional failures when they are published.
The characteristics of WinRunner are: compared with traditional manual testing, it can complete function point testing quickly and in batches; The same action can be performed on the same test script, which eliminates the misunderstanding caused by manual testing; In addition, it can repeatedly perform the same action, and the most boring part of the test work can be completed by the machine; It supports program-style test scripts, and a high-quality test engineer can use it to complete extremely complex tests, and reuse test scripts by using wildcards, macros, conditional statements, loop statements, etc. It provides a good integration and support environment for most programming languages and Windows technologies, and brings great convenience for functional testing of applications based on Windows platform.
The workflow of WinRunner can be roughly divided into the following six steps:
1. Identify the GUI of the application.
In WinRunner, we can use GUI Spy to identify various GUI objects. After recognition, WinRunner will store them in the GUI mapping file. It provides two GUI mapping file modes: global GUI mapping file and GUI mapping file for each test. The biggest difference is that the latter generates a GUI file for each test script, which can be automatically created, stored and loaded. It is recommended that beginners use this mode. However, this mode is not easy to describe the changes of objects, and its efficiency is relatively low, so the former is a better choice for experienced testers, and it only produces a GUI file, so the test script is easier to maintain and more efficient.
2. Create a test script
When building a test script, it is usually recorded first, and then the required TSL (test script language similar to C language) is manually added to the recorded script. There are two modes for recording scripts: context-sensitive and simulation. The choice is mainly based on whether the mouse trajectory is simulated or not, and simulation is generally selected when playback is needed. During recording, these two modes can be switched by F2 key.
As long as we look at the scale and function points of modern software, we can understand that the function test has already crossed the stage that can be completed by tapping the keyboard and clicking the mouse. Performance testing is an effective means to control system performance, which plays an important role in software capability verification, capability planning, performance tuning and defect repair.
3. Debug the test script.
In WinRunner, there is a special debugging toolbar for debugging test scripts. You can use single step, pause, breakpoint, etc. Control and track test scripts, and view various variable values.
4. Execute the test script in the new application.
When a new version of the application is released, we will test various functions of the application, including new functions. At this time, it is certainly impossible to re-record and write all the test scripts. We can use the existing scripts and run these test scripts in batches to test whether the old function points work normally. You can use the call command to load each test script. You can also add various TSL scripts to the call command to increase batch processing capabilities.
5. Analyze the test results
In the whole test process, the analysis of the test results is the most important, and various functional defects of the application can be found through the analysis. After running a test script, a test report will be generated. From this test report, we can find the functional defects of the application, the difference between the actual results and the expected results, and various dialogs produced during the test.
6. Return defects
After analyzing the test report, report all kinds of defects in the application according to the test process, and then send these defects to the designated personnel for modification and maintenance.
Common functional testing methods
Functional testing is to verify the function of the product, test one by one according to the functional test cases, and check whether the product meets the user's requirements. Common test methods are as follows:
1. Page link check: whether each link has a corresponding page and whether the switching between pages is correct.
2. Correlation check: Will deleting/adding one project affect other projects? If so, are these effects correct?
3. Check whether the functions of the buttons are correct: such as update, cancel, delete and save.
4. String length check: enter the content that exceeds the string length specified in the requirement to see if there is an error in checking the string length.
5. Character type check: Enter other types of content where the specified content type should be entered (for example, enter other character types where the integer type should be entered) to see if the system checks the character type and will report an error.
6. Punctuation check: the input content includes various punctuation marks, especially spaces, quotation marks and enter keys. See if the system handles it correctly.
7. Chinese character processing: enter Chinese in the system that can input Chinese to see if there are any garbled codes or errors.
8. Check the completeness of the information brought out: when checking and updating the information, check whether all the information filled out is brought out and whether the information brought out is consistent with the information added.
9. Duplicate information: enter a duplicate name or id in some information that needs to be named and has a unique name, to see if it has been processed by the system and whether it will report an error. Duplicate names include whether they are case-sensitive, whether spaces are entered before and after input, and whether the system handles them correctly.
10. Check the deletion function: in some places where multiple messages can be deleted at one time, press "Delete" without selecting any message to see how the system handles it and whether there will be any mistakes; Then select one or more messages and delete them to see if they are handled correctly.
1 1. Check whether the addition and modification are consistent: check whether the requirements for adding and modifying information are consistent, and if necessary items are added, the modification must also be recorded; Add an item specified as an integer, and the modification must also be an integer.
12. Check and modify duplicate names: when modifying, modify the items that cannot be duplicated into the existing content to see if they will be handled and reported as errors. At the same time, pay attention to whether you will report errors with your own duplicates.
13. Repeated submission form: a record that has been submitted successfully, and then submit it after you come back to see if the system has handled it.
14. Check the reuse of the back button: if there is a back button, go back to the original page, go back again, and repeat it several times to see if there is any mistake.
15. Search check: enter the contents of the system where there is a search function and check whether the search result is correct. If you can enter multiple search conditions, you can add reasonable and unreasonable conditions at the same time to see if the system handles them correctly.
16. Input information location: enter information where the cursor stays, and pay attention to whether the cursor and the input information will jump to other places.
17. Upload and download file check: whether the function of uploading and downloading files is realized and whether the uploaded files can be opened. What's the format of the uploaded file? Does the system explain the information? Check whether the system can do it.
18. Required item check: whether the system has handled all the cases where the required items are not filled in, and whether there are prompts for required items, such as adding * before the required items.
19. Shortcut key check: whether common shortcut keys are supported, such as Ctrl+C Ctrl+V Backspace, etc. , and whether shortcuts are also restricted for some fields that are not allowed to enter information, such as personnel selection and date selection.
20. Enter key check: press Enter key directly after input to see how the system handles it and whether it reports an error.