Re: nscd caching getpwent() and getgrent()

[mdm at google dot com]

>> Can you explain why?  Caching a bit more in nscd do not appear like
>> one of the stupid ideas I've read, so I wonder why it appear so stupid
>> to you.
>
> If I had to guess, I'd say that getpwent() and getgrent() are terribly
> designed interfaces that aren't threadsafe, aren't generally usable for
> systems with more than a handful of accounts accessed via nss_files, and
> are only included because POSIX requires them and ugly old code expects
> them to exist. Furthermore, they aren't ever (or certainly shouldn't be)
> used in performance critical situations.

I'll agree with all of those statements.

> Adding caching support to nscd increases complexity with
> basically no gain, especially considering that nscd is designed for fast
> random read-only access to specific datums, and not writable linear
> traversals over an entire dataset.

Compared to how complex the rest of nscd is, it's not really that much
complexity to add this. But I wouldn't say that nscd is only for fast
access to specific data - that's thinking about it like a hardware cache.
It's a software cache, so we don't have as hard a limit on the amount of
data we can cache. If it speeds something up, and the amount of memory
used is reasonable, it's a good trade.

> For the kinds of things that getpwent() and getgrent() are used for,
> you're better off using your directory service's native interface (LDAP,
> Kerberos, BigTable?, etc.) or something generic like libuser (assuming
> that has a sensible interface).

The thing we'd like to speed up is tab-completion of usernames in bash,
and bash uses the getpwent() interface to do this. Adding this support to
nscd reduces the time required to tab complete ~ from several minutes to a
few seconds. It also decreases the network traffic, even if you only do
the tab completion only once, since bash (via readline) only counts the
users the first time it walks the user list - it walks the list again to
display them.

Mike Mammarella