Re: O_EXEC and O_SEARCH

[Rich Felker <dalias at aerifal dot cx>]

On Fri, Feb 22, 2013 at 10:58:18PM -0500, KOSAKI Motohiro wrote:
> On Fri, Feb 22, 2013 at 10:17 PM, Rich Felker <dalias@aerifal.cx> wrote:
> > On Fri, Feb 22, 2013 at 10:05:03PM -0500, KOSAKI Motohiro wrote:
> >> > I'd like to have a conversation with the glibc team about O_EXEC and
> >> > O_SEARCH in the interest of hopefully developing a unified plan for
> >> > supporting them on Linux. Presumably the reason glibc still does not
> >> > have them is that Linux O_PATH does not exactly match their semantics
> >> > in some cases, and O_PATH is sufficiently broken on many kernel
> >> > versions to make offering it problematic. In particular, current
> >> > coreutils break badly on most kernel versions around 2.6.39-3.6 or so
> >> > if O_SEARCH and O_EXEC are defined as O_PATH.
> >>
> >> I'm curious why don't you implement them in kernel directly?
> >
> > See this thread for Linus's opinion on why O_SEARCH was not added:
> >
> > http://comments.gmane.org/gmane.linux.file-systems/33611
> >
> > O_NODE seems to have been renamed to O_PATH, or perhaps O_PATH was a
> > later independent implementation of the same idea; it's not clear to
> > me which happened. But the idea is that the kernel folks did not want
> > to do O_SEARCH and O_EXEC properly in kernelspace but instead wanted
> > to provide a more general flag that could be used to implement both
> > O_SEARCH and O_EXEC.
> 
> Do you mean following response?
> 
> >I suspect that what we _could_ possibly do is to have something like
> >O_NODE, and after that - if the semantics (for directories) match what
> >O_SEARCH/O_EXEC wants, we could just do
> >
> >#define O_SEARCH O_NODE
> >
> >but my point is that we should _not_ start from O_SEARCH and make that the
> >"core" part, since its semantics are badly defined (undefined) to begin
> >with.
> 
> I so, I don't think "start" mean refusing at all. However, I agree
> kernel folks dislike
> to hear "because it's posix". As far as no concrete good use case, any proposal
> may be going to be get negative response.

Yes, this is what I was referring to.

> However, if O_SEARCH is really useful, I think in kernel

Regardless of whether it's useful, it's mandatory, and you can't claim
conformance if it's omitted. With that said, I think it is useful with
the *at functions. Otherwise, the *at functions have reduced
functionality in some sense because you can't use them with
directories you don't have read access to.

> implementation is better
> because all other flags are implemented in kernel and it may prevent to create
> ugly corner case.

I agree, I really wish the kernel had just defined access mode 3 to be
O_SEARCH and O_EXEC. However, getting them to do this seems unlikely.
Even if they did do it, glibc and musl both intend to support kernels
older than 3.9, so we'd have to have some fallback mechanism for
avoiding serious breakage in apps when the kernel does not support
them right. Note that this breakage is happening ALREADY in coreutils
if O_SEARCH and O_EXEC are defined; the only reason coreutils is
working on glibc is that glibc does not define them.

Anyway, at this point (3.8 kernel, maybe some older ones too), I
believe O_PATH is sufficient to implement O_SEARCH and O_EXEC with one
caveat: If O_NOFOLLOW is also specified, fstat must be used after the
open to determine if the file descriptor obtained refers to a symbolic
link, and if so, it should be closed and failure simulated. I'm not
aware of any other cases where O_PATH would give the wrong behavior.

If we want to offer O_SEARCH and O_EXEC but avoid breakage on broken
kernels (2.6.39 through 3.5 or so), then perhaps fstat should always
be checked whenever O_SEARCH or O_EXEC is used. If it fails (which
means the kernel is broken), the file descriptor could be closed and
open reattempted in O_RDONLY mode.

Alternatively, we could define O_SEARCH and O_EXEC to the value 3, and
always do the following remapping in userspace:

1. Try to open with O_RDONLY. If it succeeds, we're done. This is
REALLY nice because it means O_SEARCH and O_EXEC "just work" even on
ancient or broken kernels as long as the target file is readable.

2. Else, add O_PATH and try again. If it still fails, we have a
pre-2.6.39 kernel and there's nothing we can do, so just report
failure.

3. If open succeeds with O_PATH, then if O_NOFOLLOW is also specified,
check fstat, and close the file and report error if fstat succeeded
and the obtained fd was a symbolic link.

4. If fstat failed, we have a buggy kernel, so either close and report
an error, or just ignore the failure (possibly ignoring the
requirements of O_NOFOLLOW), as there seems to be no way to handle it
correctly on such kernels.

If the kernel developers ever add O_SEARCH/O_EXEC at the kernel level
with our proposed value of 3, a step 0, just passing the value to the
kernel directly and seeing if it works, could also be added.

Rich