The Art Of Software Development (part 5): Adding Value - Playing The Numbers (
Page 4 of 6 )
It doesn't matter how long you've been developing software - you learn
something new with every project you execute. And so, after a project
has been formally concluded, it's important to spend some time auditing
the effort and expense that went into it, both to see if it matched your
initial estimates and to locate areas for improvement.
Some of the possible audits are:
- Requirements definition: This report measures the number of
iterations a requirements specification goes through before it is
formally approved, and provides an indication of how rapidly and
efficiently the organization can understand and document customer needs.
- Estimation accuracy: This report compares the effort and expense
estimated by the organization at the beginning of the project with the
actual effort and expense at the conclusion, and thereby measures the
organization's skill at providing customers with accurate project
estimates and the validity of the estimation methods and formulae used.
- Test planning and execution: This report compares the tests planned
with the tests actually executed in each of the testing phases, in order
to measure the integrity of the test process. The time and cost
estimates of each test may also be compared against actual time and
cost, to verify the accuracy of the organization's estimation formulae.
- Error resolution: This report graphs the number of errors reported
against the number of errors actually resolved, and thus provides a
measure of how well the organization's defect management process
actually works. A variant of this report involves graphing the number of
errors against the time taken to close them, in order to measure
response time.
- Error frequency: This report graphs the number of errors against each
component of the software architecture, and can be used to identify
organizational deficiencies in skill or programming knowledge (by
mapping each component to specific developer skill sets or
responsibilities). It also provides a measure of the accuracy and
effectiveness of the organization's test plan and test cases.
- Change request resolution: This report compares the number of change
requests against the time taken to execute them, and provides a measure
of how well the organization is set up to respond to evolving customer
needs.
This is by no means an exhaustive list - every organization has
different needs, and must develop different metrics to analyze its own
performance over time.
The data that is used to compile these reports must be collected
throughout the software development process, and reports based on that
data should be generated and provided to the project manager on a
regular basis throughout the project lifecycle. This ongoing trend
analysis allows managers to identify specific problems in different
phases of the project, and take corrective action to resolve them.
In addition to ongoing analysis, once the project has concluded, a
comprehensive set of audit reports should be built and analyzed to get a
big-picture view of the problems encountered over the project timeline.
These reports can be used to identify specific action items for
different departments of the organization, and to prevent a
re-occurrence of the same mistakes in subsequent projects.
It is also useful to perform a re-estimation of the entire project once
it is formally concluded. At this point, project managers have a very
clear idea of what exactly the software requirements are (after all,
they just built it!) and they can use this understanding to estimate how
long the project would take. This estimation should again be compared
with the original estimate delivered to the customer, discrepancies
should be analyzed to understand their source, and corrections should be
made so that future estimates are more accurate.
The ultimate goal of all this analysis: better estimation, better
implementation, better quality control. All of which leads to lower
costs for the organization and greater value for the customer.
