Test Plan ! Do we really need it!

Software Test Plan (STP) was initially required (Since 1994) by the regulated quality procedures with the objective to “establish uniform requirements for software development and documentation.” The STP demanding to verify that our testing coverage procedures will perfectly meet our requirements objectives, but can we really follow this document progress and report on its quality or this is one of those demands that we always meet but very rarely invest resources to track it. In order to meet the standard here are some key points that should be mentioned in such document
The DIDs are:
· Software Development Plan (SDP) - A plan for performing the software development
· Software Test Plan (STP) - A plan for conducting qualification testing
· Software Installation Plan (SIP) - A plan for installing the software at user sites
· Software Transition Plan (STrP) - A plan for transitioning to the support agency
· Operational Concept Description (OCD) - The operational concept for the system
· System/Subsystem Specification (SSS) - The requirements to be met by the system
· Software Requirements Specification (SRS) - The requirements to be met by a Computer Software Configuration Item (CSCI)
· Interface Requirements Specification (IRS) - The requirements for one or more interfaces
· System/Subsystem Design Description (SSDD) - The design of the system
· Software Design Description (SDD) - The design of a CSCI
· Interface Design Description (IDD) - The design of one or more interfaces
· Database Design Description (DBDD) - The design of a database
· Software Test Description (STD) - Test cases/procedures for qualification testing
· Software Test Report (STR) - Test results of qualification testing
· Software Product Specification (SPS) - The executable software, the source files, and information to be used for support
· Software Version Description (SVD) - A list of delivered file and related information
· Software User Manual (SUM) - Instructions for hands-on users of the software
· Software Input/Output Manual (SIOM) - Instructions for users of a batch or interactive software system that is installed in a computer center
· Software Center Operator Manual (SCOM) - Instructions for operators of a batch or interactive software system that is installed in a computer center
· Computer Operation Manual (COM) - Instructions for operating a computer
· Computer Programming Manual (CPM) - Instructions for programming a computer
· Firmware Support Manual (FSM) - Instructions for programming firmware devices

With so many items to follow it is almost impossible to do it without the use of ALM 2.0 tool (http://www.orcanos.com/qpack_alm_for_software_overview.htm ). In most cases we need to find good motivation to invest so much effort to meet this task.
On the practical side a test plan is made of two words Test and Plan. Two separate words but one focus, a plan for testing. Project plans are constantly adjusted around the triangle of scope, time and resources. In my mind a test plan is a sub-project plan focused on testing within the project. As the scope, time or resources change the test plan must adjust as well. The major objectives in the project plan need to me maintained and checks and balances of achievements should be reflected in the test plan. To follow the test plan is to follow the project plan or in most cases the release agenda based on list of requirements that needs to be meet by the project scoop. If we don't have a test plan or if we don't follow the test plan then we increase risk of missing a goal within the project plan and therefore risk the failure of the project. Can you cover everything? In most cases we can't cover everything in the project plan or the test plan. Therefore we need tools such as ALM 2.0 (http://www.orcanos.com/testing_tools_for_test_management.htm) that will also be able to measure the risk by not covering the key items in the STP. The main goal of testing is to address and communicate risk. We don't fix code, we don't set goals, we don't require features...we make sure that goals and features are in place and their dysfunction is addressed or mitigated. A good test plan is like a road map that's up to and contain only what is necessary. It refers to the leads or managers test plan, rather than being a duplicate with a few additional lines and changes here and there and that is just for the sake of “We were told to do so”. But if our tools allows us to defined the STP but at the same time to track the coverage matrix (http://www.orcanos.com/QPack_analytics.htm) using analytics reports. So what best practice tells us about good STP, its should provide enough detail and scope to unambiguously test an action or function (etc.). That said there is ALWAYS room to expand on them to expose corner conditions, race conditions, concurrent incompatible parameters and anything else you can think of. That's why you take notes during testing. So that you have a log to assist you in describing bugs and writing new test cases. This is why your ALM 2.0 tools should be capable to empress all the diverse activities in the project in one tool, and testing management is just one of them. Using good testing tool (http://www.orcanos.com/testing_tools_for_test_management.htm ) will allows you to build on top of your STP the test coverage and at the same time allows to reuse existing tests for different scoop of your STP.

The Blurry Line between ALM and PLM

Application Lifecycle Management (ALM) and Product lifecycle Management (PLM) have different technology management perspectives—the first focuses on software development methodology system architecture, software validation, coverage matrix and impact analysis whereas the second focuses on taking care of innovation processes by facilitating product definition collaboration. However, there is an intersection between the two systems. By empowering the consistent accessibility of one system's stakeholders to another system's data and processes, the integration between the two systems delivers benefits such as shorter time-to-market, lower manufacturing costs, and higher customer satisfaction.

Why External UI Testing Matters

Software Engineer Brian Driscoll uses an unpleasant client post-mortem to explain the importance of testing UI with external testers. Read more

http://www.brian-driscoll.com/2009/12/why-external-ui-testing-matters.html

How does technology affects product marketing?

Increasing usage of social media and web 2.0 by companies and customers has changed and also has added many roles in organizations. My question is how product management and product marketing roles have been affected by this new wave?

Testing the Limits with James Bach

This is an interesting article worth reading for those of you that belongs to the Testing domain.

http://blog.utest.com/testing-the-limits-with-james-bach-part-1/2009/12/

How you build your product vision

What factors need to be considered by architects, product managers and development teams for creating product vision?
What drives the definition of a vision that leads to a successful and competitive product?

http://www.linkedin.com/redirect?url=http%3A%2F%2Fbit%2Ely%2F3Thivl&urlhash=wxr_

Skills of a Successful Project Manager

In his book, "The Little Black Book of Project Management," Michael Thomsett identifies his version of the skillset of a successful project manager. I'm providing it here to give you yet another take on some of the key characteristi…

Those 8 qualification are excellent but it is not enough to be able to define them but there should be also methods and tools that needs to support most of those qualifications. In most cases the project manager spend a lot of time to collect the information he need in order to make a decision. In that case a decision support system will allow the project manager 1. to be alert ahead of time on project risks 2. to collaborate with all team in all locations via ALM 2.0 tools 3. to communicate with all stakeholders 4. to trace back deliveries to customer requirements 5. to transform customer request into change management system 6. to analysis the impact of a change on the overall project. ALM 2.0 tools were build to all this work and more. Take a look at this FREE tool http://www.orcanos.com/QPack_for_software.htm or any other tool that meets ALM 2.0 qualifications.

Criteria for Successful Project Management Offices

In the following artical you will find usfull information about the criteria for successful projects. Some of the important measures can be address with ALM 2.0 tools. http://pmtips.net/criteria-successful-project-management-offices/

Does standards helping pushing testers to the max

Our main concern is whether testers sometimes find themselves adhering to standards not because they believe them to be valuable on their project, but because they fear the consequences of being "non-compliant". Read more in the link.

Golden Rules for Building Great Software

Building great software has always been a tough task, but it becomes even more complex today. True to Orcanos’ commitment to excellence in the field of ALM 2.0, we have been building our QPack solution and our company in the past five years in order to help you develop great software. We have been putting a lot of effort and paying meticulous attention to every step we take, while trying to innovate in our own field.
As a result of this hard work, you can now benefit from 10 points that will help an ALM project owner lead an organization through the complete Application Lifecycle Management solution. This method has also been our own decisive success factor.
So here are ten golden rules for building great software:

1. Gain Management Skills: You can learn most of the skills needed in order to be a good manager. The management skills required include proper planning, execution and follow ups on tasks. Selection of the right steering team is also part of the management skills you need, as you will be required to empower your team to complete their tasks without your attention.

2. Have an Executive Sponsor: An executive sponsor is an executive manager who owns the ALM project in your organization but is also the one who pays the bills. Attendees who act as executive sponsors at kickoff meetings demonstrate management support for both the project leader and the ALM effort. The executive sponsor has the authority to resolve impasses that occur during the implementation project

3. Invite the Right People: Make sure all the right people attend the initial meetings. Overloading those meetings with a mix of people may create early conflicts while the first stage is all about brainstorming and rethinking your existing status. Bring subject matter experts whose advice is valuable for the discussions, as well as those individuals whose approval is mandatory in the decision making process. Make sure that decision makers not only have the authority to make decisions but also have the will to do so.

4. Assign People to Roles: Every person in the steering team is assigned to a specific role. In addition to the ALM project leader who plans and conducts the steering meetings, there is the executive sponsor, and a scribe who records decisions and unresolved issues. The ALM project owner should make sure each one of these individuals understands and plays his or her assigned role.

5. Do the Mandatory Pre-Work: It is important for the ALM project owner to ensure that everyone does the pre work prior to the actual Go-Live stage. Evaluating the solution according to pre defined success criteria is critical in order to shorten the evaluation stage. Learn yourselves first and identify the most critical issues you want to resolve. Make sure there is enough time for all the members of the steering team to review the material they need to respond to. Make sure you get feedback on critical issues first and then on the rest. Contact your steering team members periodically to make sure all decision makers are communicating well with each other.

6. Consider Pre-Meetings: For those who are new to the ALM world or to leading such a project, the ALM project owners should conduct a short training session prior to the first meeting. This will help the steering team understand the benefits of the ALM solution, the importance of the pre-work, the roles and the rules, the meeting processes and the consensus approach to decision making.

7. Obey the Rules: To ensure an orderly conduct of an ALM project, certain ground rules need to be in place. These rules are non negotiable and should be reviewed with the steering team before starting the evaluation of the ALM solution. Some of the standard ground rules are: Stick to the agenda; Discuss one topic at a time; Stop the project meetings in case a key participant leaves or is not attending; All decisions are either “yes” or “no”; Be open to all ideas; Treat all the decision makers as equal.

8. Go for Consensus: Decisions should be consensus-driven and simply be put to a vote. ALM projects involve all stakeholders in the organization, so leaving an important stakeholder out could disrupt the necessary information flow. Consensus-based decisions require participants to prefer the overall interests of the organization over individual interests. Consensus means decisions are supported both during the evaluation and after Go-Live.

9. Ensure Mutual Respect: It is important that ALM implementation will be conducted in an atmosphere of fairness and mutual respect. It is the ALM project owner’s duty to ensure that ideas and work products – rather than a specific stakeholder – are addressed in the final ALM solution. After all, ALM is an organizational platform and should be treated as such.

10. Get the Signoff: It is important that the ALM project owner will present the final deliverables (e.g. Reports, Design Documents, Plans, etc.) to the primary participants, in order to get their formal signoff. This assures the stakeholders fully accept the decisions taken and agree to move forward.

Why PMO's Fail? Read More....

On top of the following article http://www.projectsmart.co.uk/why-pmos-fail.html

I also find that what ALM 2.0 brings into this domain is what becomes more critical to the project manager respond to scenarios that can be predicted. ALM 2.0 as collaborative system does the work and get the work done.

When to say SW is ready for BETA

When, how and what are parameters that a SW under development being verified by the test office should be judged ready for BETA (customer/user acceptance). One option could be based on the timelines and business case but then should we really follow the time lines all the time and send out a crapy SW to customers...

The application lifecycle management evolution

I have received several questions regarding the evolution of ALM and ALM systems.

Following post is a brief regarding the Application lifecycle management generations and evolution

Before ALM - There were some tools, or even no tools at all, and each tool did its role. There was no synchronization between the tools.

ALM 1.0 Tools (before even the term ALM was too common) was a set of separated tools that were synced manually in the best case.

ALM 2.0 is the unified platform for all development aspects, one repository, strong analytics and sharing of common services such as security, workflows, templates, etc.

ALM 2.5 (A new term I thought of...) is the next generation of application lifecycle management tools, talks about using the data collected during the development, define behavioural patterns and proactively generate alerts and metrics that will help the organization taking the right decisions, while taking into account every aspect of the development.
Each change submitted will list all the affected areas in the system, will show relevancy rank of affected items and show quality rank for each affected item.

More on that on my next posts....

The importance of requirements management

The following post discusses the importance of requirements management:

• The requirements management is the “technical language” that collaborates customers, marketing, development and QA

• Gathering, documenting, verifying, and managing requirements throughout the lifecycle ensures satisfying customer needs

• Requiremnts management assures finding defects earlier in the development process

• Keep product definition up-to-date and

• Communicate changes to the R&D

• Ensures that iterative refinements and unanticipated changes are adequately dealt with during the developemnt lifecycle

• Reduce communication overhead with customer and reduce number of patches delivered to customers for each release

How important is to Jump Start with ALM professional service program?

It’s not easy for software companies to improve operating efficiency while responding to market changes. Improving productivity across business processes requires visibility, speed, and automation. The ALM professional services program can tailor the ALM software to your needs and roll it out quickly and, it can be adapted to changing requirements. Find out how it can grow with your company.

Whether you device manufacture industrial components or software vendor, your success depends on how quickly and effectively you adapt to change. Coping with evolving customer demands, government regulations, and your own market requires constant innovation to bring new products and services to market while controlling costs especially information technology costs.
It is a challenge to be competitive as well as efficient and profitable.
Even with a local customer base, you are likely operating in an integrated global environment where stakeholders are placed around the globe. Limited visibility into business drivers can hamper your efficiency, ALM introduce a well define concept of information gathering at the same time. Rising costs and shrinking cycle times for software development require increased internal efficiencies from flexible software development methods to streamlined order fulfillment. To improve productivity across business processes, you need visibility, speed, and automation. The ALM professional service program is designed to help you tailor an ALM unified platform solution to your needs and get it up and running quickly. Complete with preconfigured system, the program should be easily adaptable to changing requirements, growing with your business. The ALM unified platform solution supports the processes you perform to meet daily business needs from market requirements and product requirements roadmap planning to development and testing and deployment at customer site. When you have selected ALM solution to your organization, at the same time make sure you have preset resources to the implementation program which will enable your employees to make most out of the ALM solution .

140 words – ALM Analytics and reports – Optimal Requirements vs. Test creation trend

This report usually generated from ALM system that contains requirements management and test management module.

















1. Requirements definition - at the beginning of the application lifecycle, the requirements creation trend is higher, later on used by development team and QA team.
QA starts writing his test plan, but in lower rate.
2. End of Requirements definition - QA continues writing test cases.
3. Equation point 1- where number of tests equal to number of requirements.
4. Start QA cycles - start running tests in QA cycles. At this phase, we usually have more test cases than requirements to assure requirements coverage.
5. Next release starts - product manager submits requirements for next release.
6. Equation point 2 - Number of requirements is equal to number of test cases. The same application lifecycle occurs again.

140 words - 15 practical tips for test management

1. Cover each requirements with at least one test
2. Write functional tests first
3. Start with positive tests first to avoid show-stoppers
4. Expose your test plan to developers and product managers before QA starts
5. Use defects history for better coverage of existing problems
6. Write short, understandable tests.
7. Use parameters matrixes for multiple configurations
8. Convert each reported defect to test case, in case defect is not reported during test run.
9. Set effort estimation to assess the time required for test execution
10. Create execution sets for functional, sanity, regression tests
11. run test case only once in each QA cycle
12. Create reports for better tracking of your test execution
13. Learn to analyze your test management reports
14. Create understandable and clear defects
15. Use defects steps to reproduce to assure defect can be reproduced

140 words – Process flow/ Workflow in ALM systems

A work flow/process flow is a set or predefined steps that illustrate the progress of work being done.
Using workflow in application lifecycle management is used to track development progress of all ALM artifacts, such as project, requirements, tasks, test cases, defects and so on.

Most ALM tools allow the admin to set the default assignee upon ALM artifact state (status) change.

For example, a defect workflow manages the defect lifecycle, from creation to resolution.
Defect reported in status “New” and assigned to the team leader, who allocate the defect to the relevant developer. Once done, the defect status is changed to “Assigned”.
The assigned developer can fix the defect (“Fixed”) or reject the defect due to the following reasons:”Not reproduced”, “As design”, “Need more info”. In that case, the defect is routed back to the creator (Tester) for further investigation.

Full workflow examples will be given in the next posts

140 words - Implementing Application Lifecycle Management Tool

purchasing and installing the right ALM system or set of tools is not enough. Most organizations don’t realize the hidden cost of using ALM systems. Using an integrated application lifecycle management requires the organization to consider the followings:

Existing ALM methodologies might need to change;

Resistance of users, now need to “serve” another system, not to mention the fact that they are under “surveillance”, since all their activities and performance is now documented…;

It requires some time for to exploit the full benefits an ALM system;

If several tools are in use – traceability is not natural. It might require lots of investments in order to make all systems “talk” to each other;

And finally – not all organizations are mature enough, or ready for such a change, as good as it can be

140 words - The purpose of requirements management tools

The purpose of requirements management is to maximize the likelihood that an application will function as intended and deliver its projected value to the business. Forrester defines requirements management as: The storage of requirements, the tracking of relationships among requirements, and the control of changes to individual requirements and groups of requirements.

Requirements management is both a discipline and a category of tools. Business analysts, business customers, product managers, project managers, and developers use requirements management tools to increase the efficiency of their requirements management practices. The larger the development effort and the more granular the requirements are, the more important tool support is in making proper requirements management cost-effective. For this reason, firms in industries like aerospace and defense, telecommunications, and automotive have long used requirements management tools to support their embedded systems development efforts.

140 words - ALM terms

Each term will be described in separate post. But please note: This blog addresses each part of the lifecycle in a holistic way. Just defect tracking is not interesting.
ALM is a live organism. One part cannot live by itself. Each aspect is a part of the whole, and this is the way I will address it.

ALM - Application Lifecycle Management, ALM 2.0, ALM for medical, SDLC - Software Development Lifecycle Management, SLM - Software Lifecycle Management, project management, process flow, risk management, high level/low level Requirements Management, high level/detailed design, R&D tasks, unit tests, test management - Test plan, Test automation, test run, test results, defect tracking, build manager, source code integration, change management, analytics, coverage matrix, traceability matrix, release manager, tracking and reports, Regulations, The PLM/ALM aspect, 3rd party integration, customer management, action items, release notes.

140 words on ALM prediction

140 words on ALM Prediction benefits

Why do managers need prediction? Why they know more than any ALM system.
True. No ALM analytics system can replace an experienced manager.

Well, prediction is not a replacement. It’s a decision support tool.
It can tell you when you should stop the development and check if you exceed the planed release; when your QA is diverging; when customers defect reporting trend is more than the organizational average. It can help you take decisions whether to implement a change in your product; asses the impact of change in terms of time and cost; asses affect on other components in your product.

To summaries – ALM prediction is not the new sheriff. It is a decision support tool helping us to cope with the ever-changing environment we all live in, and see what’s under the ground…

ALM blog - 140 words posts

I want to thank all of you readers for the emails, and my apologies for being away for 2 weeks.
I guess application lifecycle management is really a growing trend and I see lots of interest in the ALM area, both in the Internet and during my work.

The ALM posts from now on will be as short as possible and as focused as possible.

I am going to start 140 words posts (seems it's the spirit these days…)
I will make it as interesting and as strait to the point as possible.

I will talk about a research my company does these days, in the area of Application Lifecycle Management prediction, such as predict of future defects, a change impact on overall project, estimated cost, and much more.

Sounds like science fiction?

I agree.

Well, 140 words are over…

Useful Requirements Definition

Overview
This post describes the guidelines for a superior Requirements definition

Guidelines for requirements management:

1. Make short requirements
2. Make testable requirements
3. Make precise requirements
4. Make unique requirements
5. Manage requirements traceability in all ALM aspects (management, development, QA)
6. Define requirements workflow in order to manage requirements lifecycle effectively
7. Use hierarchy to manage a set of requirements by category (functional, security, general, safety, etc.)
8. Manage relations between requirements
9. Distribute requirements in different releases

Benefits of good requirements management:

1. Assisting in customer needs traceability. Basically, the SRS (Software Requirements Specification) will provide feedback to the customer
2. Setting clear expectation between development teams
3. Preventing scope deviations
4. Reducing development efforts
5. Providing the input for the testing team, in the validation and verification process
6. The first stage of requirements definition forms the basis for the project’s effort estimation
7. Good software management decompose the problems into component parts which are easy to understand, trace and cover by other ALM artifacts

The lifecycle of requirement definition process
Requirement first revision
“User shall be able to pay in several payment terms”

The revised requirement
“System will have a payment form, and user shall be able to enter this form and select a specific payment term”

This requirement can be broken down to several sub requirements, so we will use hierarchy to manage this set of requirements:

1. HEAD: Payment Module
1.1 REQ-112: Payment form
1.2 HEAD: Payment methods
1.2.1 HEAD: Credit card
1.2.1.1 REQ-114: Pay by Visa
1.2.1.2 REQ-115: Pay by Amex
1.2.2 REQ-116: PayPal

Requirements Management - Traceability Matrix

Requirements traceability is about tracing the lifecycle of requirements in all ALM aspects. It can be established using a variety of tools including Application Lifecycle Management (ALM) software, requirements management software, databases, spreadsheets, in-house tools or even with tables or hyperlinks in a word processor.

A requirements management traceability matrix is created by associating requirements with the other ALM artifacts that satisfy them, while the relationship of driver to satisfier can be one-to-one, one-to-many, many-to-one, or many-to-many.

Following relationships form the traceability matrix:

- High level and low level requirements
- Traceability from marketing department to R&D
- Risk management – relate requirements to risks
- ALM artifact such as code, test cases, test results, defects, tasks, detailed design

The following image illustrates the different relationships valid for requirements traceability:

What is Application Lifecycle Management (ALM)

Application lifecycle management becomes more and more crucial to project success as software becomes a major part of our lives. As offshore becomes an integral part in high-tech companies, each software development company must have a full visibility and control on the development process, in terms of quality, budget, and time to market. Furthermore, there is an increasing demand for compliance mandates by customers and the on going changes require an efficient change management.

According to Forester, Application Lifecycle Management, ALM, is “the coordination of development life-cycle activities, including requirements, modeling, development, build, and testing, through: 1) enforcement of processes that span these activities; 2) management of relationships between development artifacts used or produced by these activities; and 3) reporting on progress of the development effort as a whole”

So, there are 3 main pillars combining an ALM solution:
· Traceability of relationships between artifacts in ALM
· Automation of high-level processes in Application Lifecycle Management
· Analytics to provide visibility into the progress of development efforts in ALM

ALM 1.0 is talking about a single tool for each role; Application life-cycle management tools feature an impressive amount of redundant and usually inconsistent functionality in areas like workflow, collaboration, reporting and analytics; Repository synchronization is the primary means for integrating application lifecycle tools today - even when the tools concerned are all from the same vendor, but it is often difficult to establish, costly to maintain, or flat-out unworkable; Effort spent building and maintaining synchronizations leads to No single source of truth, and leads to overspending on ALM licensing.

ALM 2.0 talks about a single platform for the coordination and management of development activities, and not a collection of life-cycle tools with limited ALM features.
The advantages of ALM 2.0 over ALM 1.0 include:
One integrated system to manage all ALM artifacts

Product packaging - provide end users with simpler, cheaper tools, as one ALM platform provides a set of natively integrated tools.

Use of open integration standards - use of Web services APIs to ease and deepen integration between ALM tools with already existing legacy systems or single ALM tools.