ArsTechnica has an excellent article about the recent Firefox 3 “blocking bugs” hubub, and the resulting response from the Firefox community. I will say that this article and this issue really sums up some of the difficulties of the software QA process – especially at the end of a cycle near release. Everyone (testers and developers) views their bugs, their issues, their code as the most important. This is difficult to overcome at times in a corporate software environment, so I can’t imagine how hard it is in an open source software environment. It’s one of the key challenges to QA: prioritizing bugs near release and determining which ones truly are blocking issues, which ones are incorrectly defined as blocking and which ones can be shipped and put off until an early service pack or hotfix.
The inflated number of blockers doesn’t reflect problems with the Firefox development process or the program itself. Rather, it indicates that Firefox community members who actively participate in bug reporting and triaging are having trouble prioritizing the bugs properly. This is a very common problem that often emerges in large open source development projects towards the end of the release cycle.
Exactly. Hell, even corporate software development faces the same issue at times.
Asking maintainers to reevaluate bug priority is a way for Mozilla to refocus development on the most important issues so that the software is as robust and usable as possible by the release date. Reclassifying less-significant blockers is a necessary QA strategy that will actually lead to a better Firefox 3 release. No software will ever be released completely bug-free, and problems that can be fixed in updates after the Firefox 3 release can and should be reclassified at this stage so that they don’t hold up more important development efforts.
This is very, very important for QA no matter who you are working for – an open source community or a corporation. End-of-cycle bug prioritizing can make or break a release. Leave too many as deferred can make users hate you. Try and fix too many and you’ll be indefinitely postponing your release (and users will hate you). You need skilled QA and Development managers to be able to identify the stuff that absolutely MUST be fixed, the stuff that sucks but we can live with for a few weeks before a hotfix, and the stuff that just doesn’t truly matter.
Speaking of hotfixes, this is something that the Ars article doesn’t really go into but I’m sure (I hope) that the Forefox development community is already working in this fashion: branch your code even before release into a hotfix project. This project will be your first round of bug fixes for your product post-release. Even before the product is shipped start deferring the “sucks but we can live with it for a few weeks” bugs to that code branch. Put developers who are not needed for the release and bug cleanup process onto the hotfix. This way you have a head start on the hotfix that will make the (hopefully few) users who are unhappy about the shipped bugs happy.
In summary, “these are not the bugs you are looking for” and I think Firefox 3 is going to be just fine. Someone at the NYT should have done more research into software QA before writing that article or at least interviewed someone involved in QA management. By the looks of it they just reworked some press releases and added a dash of OMG!!WTFXORZ!!!700BUGZ!
Post a Comment