This is the mail archive of the libc-alpha@sourceware.org mailing list for the glibc project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: SYMLOOP_MAX


> (1) Lowering the value of MAXSYMLINKS.
> 
> I don't like lowering MAXSYMLINKS to 8. Do we really need to do that?
> Why can't we leave MAXSYMLINKS at 20, and additionally define
> SYMLOOP_MAX to 8, and let the user decide which standard they want
> to use?

I don't think it helps anyone to have fuzzily-defined parameters
around.  There is no standard anywhere that says what MAXSYMLINKS
means.  Its original meaning from 4.2BSD is the same as what
SYMLOOP_MAX means in POSIX.  The only value for which that meaning is
accurate is 8, but MAXSYMLINKS has a different value.

In general, all <sys/param.h> limit-like macros are obsolete and
deprecated, in large part because they were never standardized and ISO
C and POSIX instead standardize <limits.h> (and its associated scheme
with sysconf, pathconf, and fpathconf).  I am thoroughly against any
idea that these <sys/param.h> macros are anything but obsolete and
deprecated, because it's just too confusing.  In the ways they were
originally defined and originally used (which all came from BSD),
they have a direct correspondence to C/POSIX <limits.h> macros with
different names (roughly, MAXFOO becomes FOO_MAX).  So it is natural,
and IMHO the only sensible path, to define these--existing only for
compatibility with old source code--in terms of their POSIX counterparts.

> My worry is that changing MAXSYMLINKS will break applications when
> they are recompiled with a newer glibc that sets the value to 8
> (even if that is more correct as a reliable maximum).

That is indeed a concern, but I don't know how realistic it is.  What
applications actually use MAXSYMLINKS, and what do they think it means?
I don't know off hand of any uses of it.  If it were used, I would suspect
that the writers of those uses thought it had the semantics of SYMLOOP_MAX.

Note that I make a distinction between applications actually using
MAXSYMLINKS in their sources and the effects MAXSYMLINKS has
internally in libc, which I'll discuss below.

> (2) Make sysconf work.
> 
> We need sysconf working because it's the only easy way to make this
> truly dynamic (unless SYMLOOP_MAX can resolve to a call to sysconf?).

The POSIX way is not to define *_MAX if it's not actually a
compile-time constant.  Instead, applications call sysconf.

But, "If the variable corresponding to name is described in <limits.h>
as a maximum or minimum value and the variable has no limit, sysconf()
shall return -1 without changing the value of errno. Note that
indefinite limits do not imply infinite limits; see <limits.h>."  
So if we were to conclude that this is a variable that has no definite
limit, then the status quo in sysconf is correct.

However, the wrinkle is in the specification of this particular
variable, which is (as I gave before), "Maximum number of symbolic
links that can be reliably traversed in the resolution of a pathname
in the absence of a loop."  If you consider the "reliably traversed"
wording, then this does in fact have a definite limit, which is 8.
There are scenarios in which you can traverse more than 8 symlinks (up
to 40), but the simple category of "number of symbolic links ... in
the resolution of a pathname" admits the straightforward case of
"symlink to symlink to symlink ..." in which there can be no more than
8 symlinks before you get ELOOP.  It seems to me that "can be reliably
traversed" must consider the most-constrained case, which makes the
answer a definite one and its value 8.

> (3) Define SYMLOOP_MAX.
> 
> We need a definition for this if we don't have one and because it's
> a new define we can pick the conservative and reliable value of 8.

As explained, we don't necessarily need a definition just because we
don't have one.  It's entirely proper not to have a definition if
there exists no definite limit that is known at compile time.

The further wrinkle on this is that it's not at all clear what the
natural meaning of "conservative" is in this context.  Most of the
<limits.h> parameters are things like the maximum number of a certain
kind of resource you can be sure you can use (buffer sizes, number of
threads, etc.).  The use of those by an application is to avoid trying
to use more than the system says it can rely on getting.  However,
SYMLOOP_MAX is a bit different.  If what your application is doing is
calls to 'symlink' or runs of 'ln -s' to create a particular directory
tree structure, then it is a limit like the others and being
conservative means making sure you don't create a directory structure
and then use a pathname therein that would require traversing more
than SYMLOOP_MAX symlinks.  However, if what you are doing is chasing
symlinks (like in canonicalize_file_name), then it is really a lower
bound on what you should expect is kosher when you come across it in a
directory tree (i.e. before deciding you've hit a loop and bailing
out) and being conservative means trying never to reject a directory
tree and pathname that 'open' et al would actually accept without
complaint (while still bailing out soon enough that you catch loops
reasonably quickly).

For example, the manifest behavior of canonicalize_file_name today is
that a total of 20 symlinks can be traversed but a pathname requiring
that 21 be traversed will yield ELOOP.  (I'm moderately sure there is
no way to construct a directory tree so that the kernel's pathname
resolution will have exactly that property.)  With the change on the
roland/symloop_max branch and no other changes, that behavior will
change to allowing a total of only 8 symlink traversals before
yielding ELOOP.  (That's because SYMLOOP_MAX is not defined and
sysconf (_SC_SYMLOOP_MAX) returns -1, which is what POSIX says means
there is no definite limit, so my code falls back to enforcing
_POSIX_SYMLOOP_MAX, which is 8, instead.)

When running on Linux kernels, IMHO the most desireable behavior of
canonicalize_file_name is that it not reject any pathname that the
kernel would actually accept, with a secondary and far less important
goal of rejecting quickly any pathname that the kernel would actually
reject.  (Because of the semantics of the function, there's no
possibility--outside of races with modifications to the directory
tree--of returning a result that the kernel might reject for the ELOOP
reason--perhaps as undesireable a possibility as any--because the
result uses no symlinks at all.)  That is, the most useful definition
of "conservative" in this context is to err on the side of a higher
limit.  What really matters most is just that you actually do detect
loops, which any finite limit accomplishes.

To meet the first criterion, canonicalize_file_name should use a limit
of 40 or more.  Anything less means that it's rejecting cases that the
kernel would accept.  That's the case today with the limit of 20.  Of
course, the kernel's limit might well change in the future (it seems
unlikely it would ever decrease, but it might increase).  The kernel
does not provide a way to plan ahead for that, since it does not make
this parameter available to userland in any fashion (in fact, it's not
even a single manifest constant in the kernel source: the integer
literal 40 is hard-coded in multiple places in the code).

The logic in canonicalize_file_name is too simple to meet the
secondary criterion, even if parameterized by exactly the kernel's
parameters.  The simple scenario of a->b->c->d->e->f->g->h->i is
rejected by the kernel (...->h is accepted) while /a/b/c/d/e/f/g/h/i
(where a->1, b->2, etc.) is accepted, because it has separate logic
for direct symlink->symlink cases as well as overall logic for total
number of traversals.  That lack of precision seems fine to me, as
long as canonicalize_file_name is erring on the side of accepting
pathnames the kernel would reject rather than vice versa.

So, for the uses we have in libc (only canonicalize_file_name and the
ldconfig function with the same intent), it seems to me the desireable
situation is none of the above.  Since there is no well-defined
parameter that does or can correspond exactly to what the Linux kernel
will enforce, what I think we want is another parameter that is
private to libc, let's say MIN_ELOOP_THRESHOLD.  We'd set that to no
less than the greatest number of traversals the underlying system's
pathname resolution will allow, so for Linux 40 or more.  Then the
behavior of canonicalize_file_name would be to take the maximum of
this value and SYMLOOP_MAX, and enforce only that.

On the Hurd, it's actually libc itself that enforces the ELOOP
limitation (in hurd/lookup-retry.c) and it does so using SYMLOOP_MAX
itself (as defined in sysdeps/mach/hurd/bits/local_lim.h).  So there,
MIN_ELOOP_THRESHOLD can correctly be just SYMLOOP_MAX.

>From skimming current NetBSD and FreeBSD sources, both appear still to
have the original simple logic of counting the total number of symlink
traversals in a single pathname resolution.  They limit that to 32
(4.2BSD and 4.3BSD limited it to 8).  (Seeing this, I'll probably
change the Hurd's number to 32 as well, since it's 8 now and that's
more constrained than any other current system.)  It appears that
FreeBSD's <limits.h> does not define SYMLOOP_MAX, but its sysconf
for _SC_SYMLOOP_MAX returns the MAXSYMLINKS value compiled into libc,
which is the same one compiled into the kernel (32).

The new branch roland/eloop_threshold implements what I think is a
good first step.  It makes the code use a new (sysdeps) function
__eloop_threshold to decide, rather than MAXSYMLINKS.  The generic
definition of this function uses the greater of SYMLOOP_MAX (as
determine at compile time or via sysconf) and a built-in
MIN_ELOOP_THRESHOLD, which I set to 40.  For Linux, this means that
we've changed the limit from 20 to 40 and will no longer reject with
ELOOP any pathname that the kernel would accept.  At the same time,
I've removed MAXSYMLINKS and SYMLOOP_MAX definitions for the Hurd,
while making its pathname resolution implementation obey
__eloop_threshold () and its sysconf (_SC_SYMLOOP_MAX) return that
same value--which is now 32; this means that a future libc version
could change the number and POSIX-conforming applications compiled
with 2.17 will still get the accurate limit via sysconf.

What do people think about the roland/eloop_threshold code?

What remains is to figure out MAXSYMLINKS and SYMLOOP_MAX for Linux.
What I have a solid opinion about is that MAXSYMLINKS should match
SYMLOOP_MAX exactly: if SYMLOOP_MAX is absent, then MAXSYMLINKS should
be absent; if SYMLOOP_MAX is defined then MAXSYMLINKS should be
defined using it.  I don't really have an opinion so far about whether
SYMLOOP_MAX should be defined or not; I tend toward leaving it
undefined just because that gives us the maximum flexibility for what
sysconf may return in the future and POSIX-conformant applications
(that are dynamically linked) will get the best available value at
runtime.  It's my opinion, but not a very strong one, that what
sysconf (_SC_SYMLOOP_MAX) should return is 8, on the rationale that
this is the only value that meets the POSIX definition of "can be
reliably traversed".

What I think would be best overall is if the kernel were to drop its
extra "nested_symlink" logic so there is a clear single value that has
the POSIX meaning of SYMLOOP_MAX.  (It already has other code that
detects bona fide loops and so might detect them without iterating
many times.  I don't understand the code well enough to guess when it
will succeed or fail to detect actual loops.)  And then that it
exposed this parameter to userland querying so it doesn't have to be
compiled in.  But I'm not expecting any of that any time soon, even if
we pushed kernel folks on it, which I'm not planning to do.


Thanks,
Roland


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]