Re: (toplevel) Fix dramatic breakage for ordinary crosses (related to program_transform_name)

[Michael Elizabeth Chastain <mec at shout dot net>]

Ben Elliston writes:
> If, instead, the cache were a "dot" directory that contained files
> whose filenames had the form "key=value", then test results could be
> examined or updated using atomic file system operations and there
> should be no races.

The Linux kernel does something similar for its configuration variables
(which are different than autoconf cache entries): one file per variable.
The kernel does this to achieve fine-grained dependencies on the
configuration, so that changing CONFIG_DRIVER_FOO causes recompilation
of only the files that depend on CONFIG_DRIVER_FOO.

With a little care I think this would be thread safe.  One tricky point
is that you have to be prepared for a file to not exist at the moment
you open it because some other thread is doing an "unlink / rename" at
that moment.  Since this is a cache, that will simply cause some extra
cache misses.

There is a potential for a thundering-herd performance problem where every
thread wants to compute HAVE_FOO_H, they all deposit identical copies into
the cache, and then they move on and see that HAVE_BAR_H is not in the
cache either, lockstepping their way through similar configure scripts.
It would actually improve efficiency for autoconf to randomize the order
of tests in the configure script.  :)

Also, I think that you could mix the two cache styles in a single tree.
All the old-style subdirectories would use config.cache, and all
the new-style subdirectories would use the new fine-grained cache.
Serialization would be needed only for old-style "config.cache"
subdirectories.  New-style subdirectories could somehow say to autogen
"I am new-style, I use the new fine-grained cache, don't serialize my
configure goal with other configure goals."  There would be a performance
penalty for a mixed tree, which would go away when the last old-style
subdirectory got upgraded.

Michael C
back-seat driver :)