Disabling HTTP access to git repos

[Deryck Hodge]

On Nov 26, 2007 7:35 AM, Deryck Hodge <deryck at samba.org> wrote:
>
> On Nov 26, 2007 4:54 AM, Volker Lendecke <Volker.Lendecke at sernet.de> wrote:
> > On Wed, Nov 21, 2007 at 09:50:24AM -0600, Gerald (Jerry) Carter wrote:
> > > > Gerald (Jerry) Carter schrieb:
> > > >> Folks,
> > > >>
> > > >> I've asked Deryck to disable the access to URLS at
> > > >> http://people.samba.org/git/.  This is due to HTTP's
> > > >> inefficient transport of git repositories.
> > > >
> > > > Is that really a problem?
> > >
> > > The bandwidth usage on us2 quadrupled yesterday due to
> > > multiple git-clones via http.
> >
> > Is it certain that this is not due to robots? A good
> > proportion of ftp.sernet.de's traffic used to be search
> > engines, this might be the same. robots.txt?
> >
>
> Hi, Volker.
>
> That's an excellent point.  We disallowed for bzr but didn't update
> robots.txt when we went to git.  I'll make the change to robots.txt
> now.  I'll also look at the logs again to see if we have a lot of
> traffic from bots.
>

Just double checked the logs and it was all normal clients, not bots.
The highest usage was from git clients with user agents like
"git/1.5.3".  But at least robots.txt is updated now if we do open it
up again.

Cheers,
deryck