Lessons Learned Managing Software Development in Startups - Tracking Bugs (
Page 3 of 4 )
When product development finally got underway, the need for a bug tracking system became clear. Since we were using Lotus Notes for groupware applications at that time, we decided to use BugTracker(since discontinued) from the Eden Corporation, a Notes-based bug tracking system.
We used BugTracker through the first two major release of our product (over a period of more than a year). Several problems became apparent but we stuck with BugTracker, mostly due to inertia. The problems included:
- The obvious lack of integration between CVS and BugTracker: A lot of manual labor was necessary to keep track of what was actually going into each release. Just because a feature or a bug fix was listed as complete did not mean that the code had been checked in. We have one.
- Referring to a bug became awkward since the developers (all except one) did not use Notes for email. If a developer referred to a bug, the reader would have to launch the Notes client and navigate to the bug report. Had we been a pure Notes shop, this would not have been a problem since Bugs could have been referenced with DocLinks which is a means of linking one Notes document with another. Now, a web front-end for Notes called Domino is available but its slow and the URLs are enormous. Also not all of the BugTracker facilities were available via Notes.
- Since developers used Linux for development, it became awkward to have to maintain a separate Windows machine just to run the Notes client (although VMWare does help).
- Finally, customizing BugTracker was hard since no one had the expertise in-house.
When we had some free time between releases, we decided to switch from BugTracker to
GNATS, a popular open source bug tracking system from the OSF. Overall, GNATS has been fairly good and GNATS has both a CLI and a convenient web interface. This has proved very important since the web interface makes it much easier to refer to bugs in email and documents. Another good feature is the tight integration with email. By including the proper address, an email thread about a bug or feature can be captured in the audit trail.
GNATS, like all bug tracking tools, needs to to be customized but this turned out to be relatively straightforward since the source code was available and we have good Perl expertise in-house. In addition to tracking bugs, we also use GNATS to keep track of feature requests -- as everyone is aware, the difference between a feature and bug is small indeed! GNATS is a flexible tool and scales very well. At the current time, there are several thousand bugs and features recorded in our GNATS database.
One useful local modification that we have made is to associate a Twiki document with each bug report. We use Twiki for general project management functions. This has been useful in a number of ways, the most important of which is that it allows to add well-structured design and background information with each bug report.
The biggest problems that we have had with GNATS are:
- Its very difficult to track bugs that apply to multiple releases. A bug in GNATS has a single "state" attribute. What we need is a way to track the state of the bug for each release. Thus for example, a bug may be in the process of getting fixed for one release while a completed fix may already be ready for another. There are two (unsatisfactory) ways to get around this problem:
- Add an additional release-specific attribute for each release branch. Such attributes could have names such as state_3.0. This is ultimately not scalable however.
- Use a "clone" mechanism. This is the approach that we have adopted. If a bug applies to more than one release branch, we create one bug in GNATS for the original branch in which the bug was found. We then "clone" the bug for other release branches to which it applies. This does allow us to separately track the bug state for each branch but has the disadvantage of greatly proliferating bug reports.
- While the audit trail is a good thing to have, it's not searchable. We are not able to perform queries of the type: "who wrote the code for this bug fix". This is especially annoying when a bug has been fixed by several developers.
GNATS is not a bad bug tracking system overall but, as is the case with CVS, we would probably not use GNATS if we had to do it again. GNATS works best with scenarios where there are no long-levied maintenance branches. It's clear that a proper database backed bug tracking system is needed in such cases.
/ 34 
