Sunday, October 28, 2007

[altnetconf] RE: What is your current "task system" of choice, and how tied are you to it?

I was reading a post on the alt.net mailing list that just showed up on my inbox and thought it would be best to reply here:


We use Bugzilla for tracking everything that goes on with our applications. We have tried getting rid of it and failed too many times now (to replace it with JIRA, Fogbugz, mantis, SugarCRM, SalesForce and VSTS to name a few), there is just no better application. Bugzilla is not exactly what we want, but neither is anything else. We have since gotten rid of the idea of replacing Bugzilla and have integrated it with the rest of our processes. It receives commit messages from subversion; it will soon receive error reports directly from the application; we are looking into getting ccnet build reports to integrate into it. Our customer service had been using sharepoint to track and deal with their tasks, but they will soon be switching fully into Bugzilla. Our scrum board is nothing more than a shared search that looks at bugs that block a sprint tracker (this works really well as I work from Montana and the rest of the company is in Ohio, Pennsylvania and New Jersey and an actual whiteboard would be rather impossible). We now use it to do everything from manage releases to tracking defects and enhancements to track modifications made to the firewalls and network infrastructure to record helpdesk phone calls with their solutions.

Needless to say, our company runs off Bugzilla (yes, that takes BDD to a whole new level).

Even though it is not perfect and the imperfections are glaringly obvious (yet nearly impossible to explain how to fix), wherever it is good it is almost perfect. The perfect tools for this would fade into the background and no one would realize they were using them. It is very natural to use a buglist for the scrum board. It is also quite natural to commit a revision to subversion and have a bug get resolved as a side effect (well about as natural as the act of committing in subversion can be). Or to publish a release (another commit that goes against a different bug) then get up and go get a drink as CruiseControl.NET updates the qa environment and bugzilla resolves all the bugs waiting to get to qa, sending emails to the relevant testers that the issues are ready for them to verify, all without any user interaction (other than the commit).

What isn’t good about it is that anything that isn’t integrated or doesn’t have a natural appropriate workflow from the business perspective (regardless of the tool) is very difficult to get anyone in a habit of using (time tracking for example has not caught on despite the fact that we need to do it and know we have for the past 2 years). If the item that we try to quantify with Bugzilla is not a direct result or direct byproduct that appears on its own (even though it is something that can be figured out and we know how to do so) then it inevitably doesn’t get tracked.

Time tracking is an example there, but so are:
“What UI areas does issue X affect that will be visible to end users?”
“Does this issue need to be in release notes?”
“What was the outcome of meeting N on mm/dd/yyyy which related to this bug and why was implementation A taken over B?”
“What meetings has this bug been brought up in?”
“How much impact does this issue have with end users?”

We have been trying to figure out how to quantify those questions and many like them for several years and that is why we have tried other systems.

Additionally we have run into another issue which dwarfs the questions of this topic, as ultimately I feel that any of those systems could have wound up working for us (except VSTS, that one just didn’t fit the bill at all) had they been introduced at the right time (Bugzilla was introduced early and was simply leaps and bounds better than mere email), and that question is: “How do you deal with taking over another company that is at a different location and uses an entirely different task system?” The single biggest problem with Bugzilla at our company is that it isn’t enough to receive the corporate Buy-In from the higher ups in other locations who are already entrenched in a different system. It is far more important to be using a single task system companywide which works good for most things (and excels at a couple) than it is to sit down and figure out which system is best for you right now.

4 comments:

Corey said...

Folks interested in your post - especially folks starting out on an evaluation of tasks systems, might also be interested in a blog entry of mine with links to other accounts of comparisons of
issue tracking systems .

fallout said...

Those are some nice articles, but this post wasn't meant to actually compare Bugzilla with other systems and it doesn't represent my actual point of view (I really do like Bugzilla, I think it is the best system available and it does almost exactly what I think a bug tracking system should do). I might get around to writing actual comparison posts.

If it were a comparison (from the company point of view) I would have pointed out the reasons we are using Bugzilla (for example: scalability, Bugzilla is proven to be scalable to hundreds of active users and hundreds of thousands of bugs) as well as things we did like about the other systems and issues we had with them.

For a quick idea:
JIRA - too complex and didn't feel centered on the person doing the work
FogBugz - the imposed workflow just didn't work for us
Mantis - actually pretty good, it seemed hard to find old bugs though, almost like the goal was to fix the bugs, not to keep track of them
SurgarCRM/SalesForce - too much clutter; bug tracking was not in the initial design of these products and if feels like it has been bolted on
VSTS - too technically oriented; it was too hard to make a new bug


As for your product: I don't think we are the intended users. BugTracker.NET doesn't impose enough structure for the quasi-computer-illiterate. You have to actually know you are dealing with a bug to be tracked and want to keep track of it in the system. Comparing that to Bugzilla, every "would-be-new-bug-entry-person" in the company opens their browser and upon visiting bugzilla has a very structured flow for entering a bug; they must first select the product they are dealing with (which for the most part they do know); then the person has to enter a summary and a details section. Along the way they can either accept the defaults for the component, version, status, priority and severity or change them (each of these is important to the developer who eventually gets the item, but often the creator doesn't know the desired value; note also that each of these fields is defaulted on a per product basis). Having this process run so smoothly when there is 50+ users and the most common question I (the Bugzilla admin) have to answer being "how do I log in?" (it is on the home page right in the center of the screen and it is the only thing you can do before using Bugzilla, and it remembers your login so you don't need to do it except when you clear your cookies) is so important I can't begin to measure it. BugTracker.NET is good for small projects with a handful of close nit developers who do almost free form projects but I don't think it shares the same audience as JIRA, Gemini, Mantis or Bugzilla (which clearly seem to me to be in direct competition with each other).

Regardless I would love to hear your comparison BugTracker.NET to Bugzilla (though I think it might be a lot like the Gemini one). I (as being a somewhat random contributer to the Bugzilla project) am rather interested in this class of software as well.

Corey said...

fallout - I don't have any experience with Bugzilla, so I can't compare.

For sure BugTracker.NET is aimed at an audience that overlaps with the audiences of Jira, Mantis, Gemini, Fogbugz. (I don't know Bugzilla).

It's true, that when I first created BugTracker.NET, I was aiming for just what you said, "small projects with a handful of close-knit developers". Over the years, though, people have used BugTracker.NET in settings I hadn't been thinking about, asking for changes. So, over the years those changes have accumulated, to the point that now BugTracker.NET is used comfortably in a wider variety of settings. It's very common for it to be used in companies with 50+ users, as a customer support/help desk tracker, and as a client-facing tracker (although there are things I still need to do to make it easier. That's a high priority for me...).

I think the most important features that are MISSING in BugTracker.NET that make it not fit for the biggest organization:

* Enforcement of workflow ("Only QA is allowed to set the status to 'Tested'...").

* User self-registration. Users have to be added by an admin (or project-level admin)

* Pages for managing users and projects that can comfortably handle HUNDREDS of projects, HUNDREDS of users.

The overall philosophy of BugTracker.NET will always be like that of Fogbugz: Make it as easy as possible for testers to enter a bug so that they DO enter it.

fallout said...

I agree.

Certainly your audience overlaps with Bugzilla.

I would say it overlaps almost exactly the same as Gemini does (differences being: Bugzilla is Free, runs on LAMP or WAMP and it has more documentation).

Actually, your post comparing BugTracker.Net and Gemini is what I would expect to hear you write if you compared to Bugzilla instead (well, except the things that don't match your taste; with Bugzilla's customizability all of those are solvable if they are there in the first place; basically I would expect you to replace the section with: "Bugzilla is disproportionately difficult to install and set up compared with any other tracker I have used."). This makes me want to see your comparison more in order to try to see it from your point of view (my considerations come from using Bugzilla for the past 4 years, yours would be new).