FTP

[Max TenEyck Woodbury]

tridge at samba.org wrote:
> 
> > I use a weekly mirror batch job to keep my local copy of the SaMBa tree
> > close to up to date. Needless to say it just blew its cookies.
> 
> And if you had read the mirroring page on the samba.org website then
> you would know that we ask you to use different methods to mirror, and
> to ask first.
> 
> In particular please notice this instruction, which has been
> prominently places on our site for about 3 years:
> 
>   Please do NOT use ftp mirror scripts or web robots to mirror our site.
> 
> The fact that huge numbers of people decided to mirror our site via
> ftp when we specifically asked them not to is one of the major reasons
> we had to block ftp downloads. At commercial hosting rates these rogue
> ftp mirrors cost about US$10k per year. That's a hell of a lot of
> money to waste.

Hmm. Understood. I was using the 'mirror' package that claimed to be
efficient in its bandwidth use, only copying those items that had changed.

I've dropped the request from my weekly list.

Martin Pool wrote:
> 
> Is that your way of saying you were contributing to the main machine
> being overloaded? :-)
> 
> I know there is a temptation to automatically mirror things just so
> that you have an up-to-date copy whenever you want it.  But please
> don't download files if you will not use them.  If nobody at your site
> actually rebuilds Samba every week, then mirroring it every week is
> wasteful.
> 
> If you do actually build Samba every week (good for you!), then rsync
> or anoncvs will be faster for you, and will save money for us.

I'm working on a project that does multiple rebuilds for multiple environments
automatically. It's still in the development stage, so I don't need frequent 
updates, but I do have a use for them other than just curiosity. I had asked
if I could work on the project in this list before I started using 'mirror'.
I believe that included an implicit request to make copies of the material
I needed. I got what I took to be a positive response from some people who
seemed to be in charge.

>> 2) Is there a way to get mirror to use http or are the mirrors setup to
>>    use rsync?
> 
> What do you mean?
> 
> If you go to
> 
>   http://download.samba.org/samba/ftp/
> 
> then you will be transparently redirected to an up-to-date and
> reachable mirror site.  If you find that this isn't working, please
> let us know.

I need considerably less than a copy of the whole site so I had put quite a 
few restrictions on my 'mirror' request. I'm also on the end of a very modest
bandwidth link, so the stuff I get would not be very useful to others. However
that same bandwidth limit means I have to anticipate my own needs. I was using
'mirror' because it claimed to be efficient and put less load on the source site
than ... come to think of it, I don't remember what it compared itself to. It
only knows how to 'ftp' as far as I can tell. I was asking for information about
how to get it to work properly. (See below.)

The request for limits on mirroring is less than prominent, and I'd missed it
up to the point where you called it to my attention just now. You might consider
moving the link to that notice to the top of the page, above the list of mirror
sites, rather than below it.

There is still the problem that some of the mirror sites are less than useful.

I'd have used rsync if I'd known it was available, despite the fact that I find
it a bit harder to control than 'mirror'. I'll redo my script with rsync and
only
run it when I think I'm going to need a fresh set of information for my project.

mtew