Typical industry numbers

Help Me Analyze My Prediction Results

Table of typical values by industry

Frequently Asked Questions concerning prediction results

Typical industry numbers

Measurement

Industry averages

Impacts this result

Specific organization, developmental, change control and product measures

Not correlated to specific industries. These measures are correlated to capabilities versus industry.

Defect density

Inherent defects

Failure rate

MTTF

SoftRel Full-scale score/computed defect density

The defense industry had the highest scores and lowest defect densities.

View the average defect densities for each industry by:

	Toggle the industry type by selecting the application type from the main dialog that is always displayed.
	Select Prediction->Choose model for display and select toggle the Use SoftRel Industry Average method radio button.

Defect density

Fielded defects

Failure rate

MTTF

Q - conversion ratio between defects and failures

1-3

	Very slow growth after delivery
	Not many changes after delivery
	Infrequent usage after delivery.
	Strategic defense systems.

Operational Failure rate

Operational MTTF

4-9

	Conservative growth after delivery.
	Process control
	Airborne systems

Average growth after delivery

11-20

	Aggressive growth after delivery.
	Tactical defense systems.

20+

Experimental or developmental application.

Percentage of severe defects

This is related to cost of having a defect and fixing it which may or may not be industry specific. Systems that have low Q values also tend to have low percentage of severe defects. 1% is a typical number for monetarily or safety critical systems. This may be higher for systems that can be easily serviced after delivery.

Severe operational failure rate

Severe operational MTTF

Number of hours of operation per month

This is product versus industry specific.

Operational failure rate

Operational MTTF

Language and code expansion ratio

Not industry specific.

Defect density

Inherent defects

Failure rate

MTTF

Effective size

Not industry specific

Inherent defects

Failure rate

MTTF

Number of months of growth period

This is related to how easy or available the software is to support more then the industry type. Often projects with a very low Q have a small number or even no growth period after delivery while projects with a very high Q will have a long growth period. This number is generally equal to the number of months that this version will be supported prior to another major release containing significant new features.

Operational failure rate

Operational MTTF

Operational Failure rate

These are related to required mission time rather then industry. The shorter the mission time the higher the expected failure rate can/may be.

The mission time is the average time that the software must be completely in use without serious degradation for one instance or user. Examples:

	Average number of hours that one plane is in flight
	Average number of hours that a small business owner spends on the phone for one year.
	Average number of hours that one person uses a particular software product for a whole year.

Reliability (presuming an exponential function = exp(-failure rate * mission time)

To find a typical range of values, determine the required mission time and reliability for that time and plug into the reliability formula above.

Mission time - hours of end user software operation during which the reliability objective must be met.

Reliability objective - Must be a number less then 1 and greater then 0.

Operational MTTF

This is the inverse of failure rate. See above.

Frequently asked questions

Question: The operational failure rate computed is monumental. How do I know what it should be?

Answer: You need to review these inputs for accuracy:

This may be too small or too conservative for your application at hand.

Percentage of severe defects

Review this number as shown in the above table. You may be more interested in the severe operational failure rate then the failure rate of ALL defect types.

Number of hours of operation

Review this number for accuracy. This should be only the amount of time that the software is operating and not necessarily the calendar time. You may need to use the "components" feature if each software configuration item has dramatically different duty cycles.

Effective size

Make sure that you include only the size of the newly developed software for one delivery. Do not include existing code, commented code, reused code or code that has been purchased and has already been tested.

Language/code expansion

Make sure that the right one is selected. The wrong value will multiply the predicted defects to a noticeable level.

The model used to predict defects or the inputs to the model.

	If you are using a one parameter model and have exhausted all of the above, you may need a model with better accuracy such as the short cut, full scale or Rome labs model.
	If you are using a multi-parameter model review the inputs for accuracy by reviewing the criteria for each yes answer that is contained in the short.hlp, full.hlp and rl.hlp technical help files. (You can access these from the help menu also.)
	If you are using the historical model, make sure that:

	the historical defect density was properly calibrated for language, unit of measure and differences in development practices
	the historical project was a fielded project
	the historical defect density was measured after delivery and not during testing
	the historical project had at least 95% of it's extrapolated defects discovered by delivery date (see the historical model wizard)

Once you have done the above, determine a reasonable range for failure rate by using the steps shown in the Operational Failure rate row of Table 1 above.

Question: I am getting 0 predicted defects as a result.

Answer: These are the most common causes:

Effective size is zero

	Make sure that you have the size entered AND that you have selected the unit of size that is appropriate in the Prediction->Choose model for display dialog.
	If you select function points in the Choose model for display dialog and then enter KSLOC values that will cause this problem.
	If you check the "use components" flag on the Prediction->Show results for selected model and you don't enter any size values in any of the components that will cause this problem.

Predicted defect density is zero

This can happen if you have selected the historical model and then input a historical defect density of 0. You need to change this as a defect density of zero is not valid. The other models do not allow 0 for a valid output for defect density.

Question: When I view the failure rate or MTTF growth plots under the prediction menu, I see flat lines

Answer: These are the most common causes.

You have 0 months input for number of months of growth period	Go to prediction->Show results for selected model. Make sure that something greater then 1 is input in this field.
You have input 0 for duty cycle - number of hours of operation per month.	Go to prediction->Show results for selected model. Make sure that something greater then 0 is input in this field.
You have 0 predicted defects after delivery	See table 3 for instructions on fixing this.