More and more developers are using development tools produced by the open source community like JUnit, MySQL, Eclipse, PHP or JBoss. A recent poll asked to compare the quality of open source and commercial software development tools. And the winner is... not obvious ;o)

Open source versus commercial tools 2006 (2004 answers)
Same quality: 38% (32%)
There is no easy answer to this question: 22% (24%)
Superior in quality: 20% (26%)
Inferior in quality: 12% 13%
I do not use open source tools: 6% (4%)
I do not use commercial tools: 2% (1%)
Participants: 524 (312)

Source: http://www.methodsandtools.com

For many participants, there is no difference in perceived quality between open source and commercial tools for software development. For 22% of the participants, it was difficult to give a precise answer. Diversity exists in both worlds and it is not easy to give a clear indication when you have experiences giving opposite indications. Things have not changed a lot since our 2004 poll, even if the usage of open source tools has surely increased in the mainstream development shop. This may be a reason of the 6% decline in percentage in the "OS software is better than commercial" category, as open source has been more used and could revealed some limits.

The claim that open source software is as good as commercial one seems easy to understand. Besides their open source label, there is little difference in the available support infrastructure between products like JBoss, PHP or MySQL and their commercial competitor. Backed by large companies like IBM, products like Apache or Eclipse will surely receive more testing than a small project in SourceForge. For open source software development tools, a large user base also increases the probability that associated professional services are created to provide commercial support and that the quality of the software is "commercially" managed.

For 20% of the participants, open source development tools are superior in quality to commercial ones. Besides the results of our informal pool, there have been some studies to compare the quality of open source and commercial products. Part of these studies have investigated a claim by many open source software advocates that their code quality was higher. Peer review and the amount of feedback from users are quoted as allowing open source software to achieve high quality results. When it decided to release some software in the open source world the NASA gives "to increase NASA software quality via community peer review " as its first motivation (see references). But if the size of the development team and a smaller user base could be a problem for small vendors, larger commercial organisations could also have implemented internal peer review and they have also a user community with adequate feedback channels. So why could the feedback loops and quality perception be better in the open source community?

Several factors could influence this perception:

- Developers and users (not customers!) have a higher sense of product's ownership. They feel that they both contribute to something special and it is not "just a job" or "just a product"

- The relationship between users and developers are less confrontational because
a) money is not the matter
b) expectations are often different: the product is "younger" and... there is not a marketing organisation sometimes over-selling the benefits ;o)
c) open source organisations seems to have a better responsiveness to customers request/bugs as the process is more collaborative than confrontational

Some references on the quality of open source software:

http://opensource.arc.nasa.gov/
http://scan.coverity.com/
http://www.artima.com/weblogs/viewpost.jsp?thread=21730
http://www.cyrius.com/publications/michlmayr_hill-reliance.pdf
http://opensource.mit.edu/papers/michlmayr_hunt_probert-quality_practices_problems.pdf
http://www.theglobeandmail.com/servlet/story/RTGAM.20060420.gtflkhaledapr20/BNStory/Technology/einsider

Thanks, Martin ... an interesting read (with a discreet plug for your methods and tools site   )

---