My boss once told me in passing a very insightful rule of thumb regarding HW system bugs. He called it “the rule of tens;” for every X users of your product, there will be log10X bugs or systems flaws found, with 1 recall for good measure. After a brief discussion today we concluded that lnX would work too. The idea is that if X grows linearly, the “bugs/flaws found” curve will be logarithmic. Remember, this is a rule for the whole system.
In my opinion, this trend applies equally well to software, just with a constant multiplier, i.e. C*lnX. Notice that this “rule” does not indicate how many bugs are in the product, just how many will be found.
Recently, I’ve heard a lecture by Gary McGraw from Cigital at Stanford; he went through several slides and many words to describe the gist of the above rule. For example, certain OSs (namely, Macs and Linux) assertain from the absolute number of bugs discovered that their SW is more secure. Bogus. It is directly proportional to the number of users. Notice that as Macs and Linux gain market share these days, there is more ado about their security holes. Same goes for Firefox, originally touted as super-secure, but as it becomes more popular, more flaws are found. Conclusion, more users = more bugs found.
On a related note, I love CRI‘s presentations; I have not had the fortune to hear Paul or Ben present, but I am guessing their talks are as good as their foils. In some of them they have a great slide that hints at the cause of the poor state of production software these days. And yes, it somewhat applies to hardware as well (# of transistors in the slide), but the rigorous testing of ICs is no match to the comparatively low coverage in SW. The nature of bugs in HW and SW is not really comparable; SW vendors knowingly release their buggy product (“we’ll patch it later”,) IC vendors do not (well, with errata.) In addition, SW rarely matures.
Source is here, slide #3.
FYI, log10(x) = ln(10)*ln(x), and, in general, the base b logarithm of a number x is equal to the natural logarithm of b times the natural logarithm of x, so any logarithm is the same as any other, up to a constant multiplier. Also, when you say that the number of bugs discovered is “directly proportional” to the number of users, you are stating that NumberOfBugs = NumberOfUsers*ConstantOfProportionality. What you mean to say is, “the number of bugs discovered is proportional to the logarithm of the number of users”.
Close John. log10(x) = ln(x)/ln(10). Division, not multiplication. So logb(x) = ln(x)/ln(b).