[clug] Anti-Virus Software

[Daniel Pittman]

Paul Wayper <paulway at mabula.net> writes:
> On 06/27/2010 12:39 PM, Daniel Pittman wrote:
>> Paul Wayper <paulway at mabula.net> writes:
>>> On 06/25/2010 01:16 PM, Kevin Pulo wrote:
>>>> On Fri, Jun 25, 2010 at 12:24:41PM +1000, Paul Wayper wrote:
>>
>> [...]
>>
>>> Open Source Software makes an additional assertion: that everyone can
>>> inspect it freely.  This has proven to reduce the chance of really obvious
>>> backdoors slipping into the code, and increases the quality of the code
>>> because more people see different problems and because shoddy code is
>>> exposed quicker.
>>
>> Hey, excellent.  I would be really interested, if you could point to them, to
>> see the studies that prove that OSS has reduced the chance of backdoors
>> getting in, and that it improves code quality.
>
> I'd have to find them :-) I'm sure there are studies either way, so we could
> cherry-pick our results to suit our arguments.

Well, if you turn them up I would be very interested.  It is always good to
have more information to base opinion on.

>> I am interested, in large part, because the studies I am aware of seem to
>> keep coming up with approximately the same bugs-per-line metrics as closed
>> software, and so on, so I don't feel comfortable personally making a claim
>> that OSS is any better than CSS.
>
> 'Bugs-per-line' and many other metrics have never been good ways to measure
> security, or even quality.

Do you have a proposal for a better metric?  As far as I can see that is going
to be about the most effective metric for code quality you can come by: if
this group of developers write N lines of code, they generate M bugs in the
process.

> Microsoft loves to quote "bugs in a standard install" because most distros
> install heaps of stuff (e.g. PDF readers, daemons, NTP servers, etc).  Red
> Hat and so forth like to point to the number of critical vulnerabilities in
> core systems, since that's always higher for Microsoft.

Restricting the discussion to "critical vulnerabilities" might change the
picture (though I would bet not), but may or may not actually be much related
to code quality. :)


> My claim (let's say) is based on these observations from my own first-hand
> experience:
>
> * Developing code that's viewable by others puts a lot of social pressure on
>   the developer(s) to do the right thing - pride in making something good,
>   and fear of releasing something shameful.

...and are you arguing that unknown third parties viewing your code is more
embarrassing or likely to induce social pressure that the folks you work with,
and who you are going to be looking in the eye at lunch every day for the next
week?

> * Having the source code there to view makes it easier for security
>   researchers to review and improve it.

True.  The question, relevant to the discussion, is: does this result in
higher quality and/or more secure code at the end of the day?

I don't know; my general impression, to date, is "not particularly".

> * Open Source projects often invite contributions and comments from interested
>   people while still vetting those contributions.
>
> * FOSS projects are less bound by the constraints of budget, release dates and
>   advertised claims.  They can release something when its ready, not when
>   marketing says they should or when the programmer budget runs out.
>
> Sure there are similar claims that closed source software developers make as
> to how their development model is better.  But they don't include sharing
> the code with everyone.

...yup.  What isn't proved by that is that sharing the code does make things
better.  I mean, I would love to think so, but I keep not finding evidence to
support that desired belief.


>>> Proprietary software can never make this claim.
>>
>> Sure it can; one of the most trivial ways is this:
>
> Ah, well when you say "...they may not be /supportable/ claims..." I think
> you shoot yourself in the foot there.  I mean, if we're not fettered by the
> shackles of truth or justification then it's rather easy to make any claim
> we like.

Oh, psh.  You could have said exactly the same thing about your pro-FOSS
arguments, and for exactly the same reasons: anyone can claim anything. :)

Anyway, I would like to believe that they can't be supported.  I strongly
suspect, though, that both closed and open software is approximately equal
quality.

If you wanted an easy win, though, you would point to the IETF protocol
development model vs the innards of Samba (*cough* printing *cough*), or at
CSS, A5/1, and the DECT version of the same, compared to AES and other openly
developed cryptographic algorithms.  (Not their implementations, however. ;)


> And all the usual arguments - such as yours, and professionalism and funding
> and resources and patents and IP and so forth - have all been put forward to
> justify closed source development.  None of them attempt to counter the "all
> bugs are shallow" and "security through openness" arguments that are the core
> of secure open source development.

...on the other hand, neither has there been much proof that those arguments
are true either: vulnerability levels across closed and open software tend to
be roughly equal, so far as I can see; the occasional study that floats past
BugTraq looking at it certainly seems to support that.

[...]

> But closed source _cannot_ claim to make its code available in the way open
> source software does.

Absolutely true.

> Closed source _cannot_ claim to be as well scrutinised by as many people.

I disagree with this second point: I can think of several examples of closed
code that are much better scrutinised than their open alternatives, despite
the availability of the source.

In this case, OpenVPN, and the Microsoft PPtP VPN.  The later are well
scrutinised by experts who do this professionally; the former still hasn't had
a real security review, or even a real protocol review, as far as I can
determine.[1]

(Ironically, of course, the worst case is that OpenVPN could be as insecure as
 PPtP turned out to be in practice.  The lesson here, kids, is never design
 cryptographic protocols.  Use an existing, tested one instead.)


> Closed source _cannot_ claim a "security through openness" mindset that
> dates back the the earlier locksmiths.

Are you aware of just exactly how much horror these is in the lock-smithing
community that their secrets — that the design of their locks, the books and
other trade secrets — are escaping to the Internet?

Seriously, many of the people in the trade are absolutely horrified at the
idea that this information is leaking out.  A large part of that comes down to
an institutionalized security-by-obscurity model.


I do take your point, though, even if your example misses: openness can breed
extra security, and the worst case is usually that you are as bad as the close
version, seldom worse.

> Those are the claims that I am saying that closed source software cannot
> make, and despite companies trying to distract us with "oh, we're so good we
> don't _need_ scrutiny" arguments they will still never be able to make those
> claims.

The claim, typically, is not that they don't need security: it is that because
of the additional commercial pressures on the company, and because of the
financial arrangement surrounding the people working on the software, they
*DO* subject the code to more scrutiny.

Which, of course, begs the question of "...compared to what", since any claim
comparing to "FOSS" rather than "a specific FOSS project" is inherently
flawed, but the general position isn't wrong.


> If everything is equal - if there are the same proportion of evil commie
> mutant traitor bad guys in the Linux kernel development team as there are in
> Microsoft's kernel development team - then the claim still stands.
>
> There are plenty of things to criticise Open Source Software about.
> But being open to inspection isn't IMO one of them.

No, I agree with that.  OTOH, I don't agree that FOSS is likely to be less
bug-prone, higher quality, or more secure as a natural result of the openness.

Those attributes come from the culture around the project, and from having
leadership that value security, or quality, as well as the ability to both
inspire and enforce code to meet those standards.

Some FOSS projects have that, some closed projects have them, and most
projects (closed, open, or whatever) never do.

        Daniel

Footnotes: 
[1]  Peter Gutmann did glance at it and make, in passing, approving noises
     some years back.  He added the caveat that he did *NOT* review the design
     soundly, let alone the implementation.

-- 
✣ Daniel Pittman            ✉ daniel at rimspace.net            ☎ +61 401 155 707
               ♽ made with 100 percent post-consumer electrons