Test Plan ! Do we really need it!

Software Test Plan (STP) was initially required (Since 1994) by the regulated quality procedures with the objective to “establish uniform requirements for software development and documentation.” The STP demanding to verify that our testing coverage procedures will perfectly meet our requirements objectives, but can we really follow this document progress and report on its quality or this is one of those demands that we always meet but very rarely invest resources to track it. In order to meet the standard here are some key points that should be mentioned in such document
The DIDs are:
· Software Development Plan (SDP) - A plan for performing the software development
· Software Test Plan (STP) - A plan for conducting qualification testing
· Software Installation Plan (SIP) - A plan for installing the software at user sites
· Software Transition Plan (STrP) - A plan for transitioning to the support agency
· Operational Concept Description (OCD) - The operational concept for the system
· System/Subsystem Specification (SSS) - The requirements to be met by the system
· Software Requirements Specification (SRS) - The requirements to be met by a Computer Software Configuration Item (CSCI)
· Interface Requirements Specification (IRS) - The requirements for one or more interfaces
· System/Subsystem Design Description (SSDD) - The design of the system
· Software Design Description (SDD) - The design of a CSCI
· Interface Design Description (IDD) - The design of one or more interfaces
· Database Design Description (DBDD) - The design of a database
· Software Test Description (STD) - Test cases/procedures for qualification testing
· Software Test Report (STR) - Test results of qualification testing
· Software Product Specification (SPS) - The executable software, the source files, and information to be used for support
· Software Version Description (SVD) - A list of delivered file and related information
· Software User Manual (SUM) - Instructions for hands-on users of the software
· Software Input/Output Manual (SIOM) - Instructions for users of a batch or interactive software system that is installed in a computer center
· Software Center Operator Manual (SCOM) - Instructions for operators of a batch or interactive software system that is installed in a computer center
· Computer Operation Manual (COM) - Instructions for operating a computer
· Computer Programming Manual (CPM) - Instructions for programming a computer
· Firmware Support Manual (FSM) - Instructions for programming firmware devices

With so many items to follow it is almost impossible to do it without the use of ALM 2.0 tool (http://www.orcanos.com/qpack_alm_for_software_overview.htm ). In most cases we need to find good motivation to invest so much effort to meet this task.
On the practical side a test plan is made of two words Test and Plan. Two separate words but one focus, a plan for testing. Project plans are constantly adjusted around the triangle of scope, time and resources. In my mind a test plan is a sub-project plan focused on testing within the project. As the scope, time or resources change the test plan must adjust as well. The major objectives in the project plan need to me maintained and checks and balances of achievements should be reflected in the test plan. To follow the test plan is to follow the project plan or in most cases the release agenda based on list of requirements that needs to be meet by the project scoop. If we don't have a test plan or if we don't follow the test plan then we increase risk of missing a goal within the project plan and therefore risk the failure of the project. Can you cover everything? In most cases we can't cover everything in the project plan or the test plan. Therefore we need tools such as ALM 2.0 (http://www.orcanos.com/testing_tools_for_test_management.htm) that will also be able to measure the risk by not covering the key items in the STP. The main goal of testing is to address and communicate risk. We don't fix code, we don't set goals, we don't require features...we make sure that goals and features are in place and their dysfunction is addressed or mitigated. A good test plan is like a road map that's up to and contain only what is necessary. It refers to the leads or managers test plan, rather than being a duplicate with a few additional lines and changes here and there and that is just for the sake of “We were told to do so”. But if our tools allows us to defined the STP but at the same time to track the coverage matrix (http://www.orcanos.com/QPack_analytics.htm) using analytics reports. So what best practice tells us about good STP, its should provide enough detail and scope to unambiguously test an action or function (etc.). That said there is ALWAYS room to expand on them to expose corner conditions, race conditions, concurrent incompatible parameters and anything else you can think of. That's why you take notes during testing. So that you have a log to assist you in describing bugs and writing new test cases. This is why your ALM 2.0 tools should be capable to empress all the diverse activities in the project in one tool, and testing management is just one of them. Using good testing tool (http://www.orcanos.com/testing_tools_for_test_management.htm ) will allows you to build on top of your STP the test coverage and at the same time allows to reuse existing tests for different scoop of your STP.

Why External UI Testing Matters

Software Engineer Brian Driscoll uses an unpleasant client post-mortem to explain the importance of testing UI with external testers. Read more

http://www.brian-driscoll.com/2009/12/why-external-ui-testing-matters.html

Testing the Limits with James Bach

This is an interesting article worth reading for those of you that belongs to the Testing domain.

http://blog.utest.com/testing-the-limits-with-james-bach-part-1/2009/12/

Does standards helping pushing testers to the max

Our main concern is whether testers sometimes find themselves adhering to standards not because they believe them to be valuable on their project, but because they fear the consequences of being "non-compliant". Read more in the link.

When to say SW is ready for BETA

When, how and what are parameters that a SW under development being verified by the test office should be judged ready for BETA (customer/user acceptance). One option could be based on the timelines and business case but then should we really follow the time lines all the time and send out a crapy SW to customers...

140 words - 15 practical tips for test management

1. Cover each requirements with at least one test
2. Write functional tests first
3. Start with positive tests first to avoid show-stoppers
4. Expose your test plan to developers and product managers before QA starts
5. Use defects history for better coverage of existing problems
6. Write short, understandable tests.
7. Use parameters matrixes for multiple configurations
8. Convert each reported defect to test case, in case defect is not reported during test run.
9. Set effort estimation to assess the time required for test execution
10. Create execution sets for functional, sanity, regression tests
11. run test case only once in each QA cycle
12. Create reports for better tracking of your test execution
13. Learn to analyze your test management reports
14. Create understandable and clear defects
15. Use defects steps to reproduce to assure defect can be reproduced