SQT - Software Quality and Testing Revision (Bournemouth Uni)
SQT
TDD
Advantages
Greater productivity
Lower fault rate
Automatic regression testing
Ensures that tests get run
Requirements thoroughly examined before code is written
Disadvantages
Longer development cycle
Training costs
Extra code produced through unit tests
Potential false sense of security
No functional testing
Cannot test interactions with other systems
Development Cycle
Write test case & unit test
Add unit test to test suite
Run test suite, ensure new test fails
Write rough code to satisfy test
Rerun test suite
Refactor unit test/code to higher quality
Later, rinse repeat for other features
Test Driven Development
Part of XP
"Test first" methodology
Only implement what's necessary - anything else may be supurfluous
Automated Testing
Advantages
Enables simple regression testing
Increases testing speed after initial test creation
Ensures that tests are consistent
Disadvantages
Code in automated tests can also contain errors
Difficult to perform functional testing automatically
Methods
Unit testing
Static analysis (built into build system)
GUI Capture/Playback tools
Load testing tools
Automated test data generation
Software Reviews (Fagan Inspection)
Advantages
Catches errors early in dev cycle
Increases quality of software
Fixing error before maintenance period 10 - 100x cheaper
Reviews can be performed on any artefact in development process
Disadvantages
Inspections take time
Training costs
Places more emphasis on logic than I/O anomolies (Myers, 1978)
Review Process
Planning - Agree meeting and materials to be inspected
Overview - Material overview and role assignment
Preparation - Materials inspected
Meeting - Material defects discussed
Rework - Author corrects errors as agreed in meeting
Follow-up - Verification that all defects are corrected.
Roles
Author - Person whose material is under review
Reader - Paraphrases the material
Tester - Reviews material from a testing PoV
Moderator - Facilitiates the meeting
Should not be a manager
Should not be a tester
Strucural & Functinonal Testing
Stuctural (Whitebox)
Static Analysis
Automated tool analyses code without running it
Static analysis by humans is Software Review
Performed on source or object code
Looks for common coding mistakes
Statements that will never execute
Null reference pointers
Uninitilised/typed variables
Unit Testing (See System, Integration and Unit Testing)
Code Review (see Software Review)
Functional (Blackbox)
Equivalence Partitioning
Break each input domain into its own partition
Define valid inputs for partition
Define invalid inputs for partition
Select 1 valid input for each test, and 1 invalid input
Each valid/invalid input should be as good as another, "Equivalent"
Boundry Analysis
Errors tend to happen near special numbers
0, 1, values of 2^n, Upper and lower limits
Test +/- 2 from each boundry
Error Guessing
Estimated by Ntafos, 1998 to be as cost effective as partitioning
Relies on picking random data from input domain
Influenced by a testers experience
Useful to supplement other techniques
Unit, Integration and System Testing
Unit Testing
Performed on small chunks of code
Uses stubs/drivers to simulate I/O
Easy to automate
Whitebox testing
Integration Testing
Tests how different modules communicate
Large portion of errors occur at interface points
Whitebox testing
Very thorough testing of modules completed early on
System Testing
Often blackbox testing
I/O presented as an end user
Performance/Load testing
User Acceptance Testing
Beta Testing
Measuring Quality
Quality Metrics
Functionality
Performance
Reliability
Usability
Efficency
Maintainability
Security
When is software ready?
Cost vs. Quality
Nigh on impossible to create faultless software
The cost of finding bugs raises exponentially
All bugs are not created equal
Bugs must be assigned priorities
A spelling mistake is vastly less imortant than frequent crashes
Cost in terms of resources, time to market etc.
Cost per bug found poor metric. Programs with more bugs look better
Average of 28.7 major problems/KLOC (Gib & Graham)
What constitutes an error?
"A defect is an instance in which a requirement is not satisfied" - Fagan, 1986
"An unsatisfactory program execution is a 'failiure' pointing to a 'fault' in a program, the result of a 'mistake'" - IEEE
Phases of software testing
Specification -> Client Review
Design -> Manual review against specification
Implementation -> Unit/Integration Testing
Product -> System Testing
Debugging
"Debugging is the process of finding the exact location of the error within the program"
