|
| |
These are the steps in the software reliability assessment.
-
Step 1 – Complete detailed survey based on interviews and artifact
reviews These questions pertain to ability to execute a project as well
as several projects simultaneously, development techniques, organization,
people, industry/application characteristics, project management and
planning, requirements, design, code, unit testing, systems testing, quality
assurance, change control, corrective action, defect tracking and metrics.
Other assessments focus only on the process. This assessment
investigates not just the process but what software engineers are doing and
how well they are doing it. Each of the questions on this interview
form has been mathematically correlated to software defects via a
proprietary prediction algorithm that has been used in industry for the last
decade. This algorithm maps software engineering practices to a reduction
of defects and does so based on FACTS. This algorithm was developed by
measuring ACTUAL fielded defects (escapes) at more than 115 real software
companies and correlating those escaped defects with more than 600 software
practices/characteristics. For more information see the attached white paper
"The Truth about Software Reliability" which is attached in "truth5.pdf".
-
Step 2 -Predict
-
Step 2A - Once the benchmarking survey is completed, the Frestimate
software determines a score and then uses that score to determine a
percentile group. The 7 possible percentile groups are world class,
very good, good, average, fair, poor, and ugly which are quantitatively
associated with the 1, 10, 25, 50, 75, 90 and 99% groups.
-
Step 2B – You can compare your prediction with those in our database and
filter by industry, SEI CMMi level etc.
-
Step 2C –The predicted percentile is used to predict the normalized
defect density which can then be multiplied by code size to predict
defects. The defect density prediction results and percentile will be
used to benchmark the software engineering practices with industry and
to establish the key strengths and weaknesses for moving forward. The
predicted defects are then computed by multiplying the predicted defect
density by the effective size. From the predicted defects the failure
rate, MTTF, availability and reliability are then computed.
-
Step 2D – Extrapolate actual reliability metrics from test data. The
“predictors” predict reliability before the code is even written. Once
the software is in a testable state the reliability metrics can be
generated using test data instead of empirical data. Frestimate
Manager’s edition has the WhenToStop module which automates the defect
data collection as well as the reliability growth models. If you have
ever used Casre or Smerfs you will find that the WhenToStop module is
much easier to use.
-
Step 3 – These analyses are covered by
SFTA and
SFMEA and
software RCA consulting services.
-
Step 4 – Improve
-
Step 4A. – Identify of key strengths and gaps from a software
reliability standpoint. The survey was designed to predict defects
and defect density, however, it was also designed as a means to identify
key strengths and gaps in the software product, process and
organization. Additionally, it was also developed to determine the
return on investment of addressing particular gaps and the relative cost
and time to implement required.
-
Step 4B – Identify gaps that should be addressed now versus
later. Some gaps cannot be addressed immediately because they have
prerequisite gaps that need to be satisfied first. The software can
identify these for you.
-
Step 4C - The Frestimate software can help you to identify the
practices that will improve that percentile with the lowest cost and
calendar time and fewest prerequisites. So, if the resulting percentile
is average percentile, you can use the software to identify the
practices that will achieve a good percentile with the least cost and
calendar time while preserving the prerequisites for each practice that
must be met for the practice to be effectively executed.
|
