This is the mail archive of the
cygwin-talk
mailing list for the cygwin project.
Re: Mirroring and stage and tmp dirs
On Sun, 16 Apr 2006, Robert Pendell wrote:
> Igor Peshansky wrote:
> > On Sun, 16 Apr 2006, Robert Pendell wrote:
> >
> >> Should we be ignoring these folders during mirroring? I was mirroring
> >> them but recently started getting permission denied errors during
> >> transfers for those folders so they are now excluded from rsync
> >> transfers (i.e. not on my public mirror). What is the stage folder used
> >> for?
> >
> > The /stage folder is used to build up parts of the changed directory
> > structure and transfer them over in one swell foop (to avoid a race
> > with the upset cron job, where it could catch the uploader in
> > mid-update and result in a broken setup.ini).
> >
> > In short, you should not mirror that directory, as nothing in there is
> > going to be useful on a mirror.
> >
> >> rsync: send_files failed to open "/stage/tcp_wrappers-7.6-2.sh" (in
> >> cygwin-ftp): Permission denied (13)
> >> rsync: send_files failed to open "/stage/tcp_wrappers_7.6.tar.gz" (in
> >> cygwin-ftp): Permission denied (13)
> >>
> >> And from earlier on...
> >> rsync: send_files failed to open "/stage/monotone-0.26-1.sh" (in
> >> cygwin-ftp): Permission denied (13)
Heh, I just looked closely at the names of the files... Looks like
someone unpacked a source archive there -- not part of your normal "create
files/directories and move them to /release" sequence... Don't know why
this happened, but my explanation below is probably not correct for this
particular case.
> > You probably caught the uploader in the act of moving files from
> > /stage to /release.
> > Igor
>
> Ok. I updated the cron job last night to exclude those folders from now
> on. I've been trying to optimized the command so it causes as few 404's
> as possible. It is setup to delete files after getting the new ones but
> there are still a few occuring.
>
> Oh and an interesting statistic. So far this month I have had more hits
> from people using Wget then the regular setup program. _Alot_ more hits.
>
> Wget (various versions) - 76841
> Cygwin Setup - 26035
>
> This is versus last month where Cygwin and Wget was very close. I
> wonder if people are starting to branch (2nd tier) off my mirror...
I doubt it. Can you also check whether the hits are for the current
binary tarballs, or for the older source/binary ones? If the former, it
might indicate the more prominent usage of clean_setup.pl (which builds
lists of wget commands). If the latter, people may be doing what I
occasionally do -- grabbing the whole older tarball to extract an older
version of some file... Also, are some tarballs hit more than others?
Igor
--
http://cs.nyu.edu/~pechtcha/
|\ _,,,---,,_ pechtcha@cs.nyu.edu | igor@watson.ibm.com
ZZZzz /,`.-'`' -. ;-;;,_ Igor Peshansky, Ph.D. (name changed!)
|,4- ) )-,_. ,\ ( `'-' old name: Igor Pechtchanski
'---''(_/--' `-'\_) fL a.k.a JaguaR-R-R-r-r-r-.-.-. Meow!
"Las! je suis sot... -Mais non, tu ne l'es pas, puisque tu t'en rends compte."
"But no -- you are no fool; you call yourself a fool, there's proof enough in
that!" -- Rostand, "Cyrano de Bergerac"