Re: Strange errors running gcc tests on Cygwin

[Brian Inglis <Brian dot Inglis at SystematicSw dot ab dot ca>]

On 2017-03-07 22:18, Daniel Santos wrote:
> On 03/07/2017 06:36 PM, David Billinghurst wrote:
>> On 8/03/2017 10:25, Daniel Santos wrote:
>>> My concern is with the dynamic portion of this behavior -- what
>>> is affected by environment variables.
>> Many years ago I ran a nightly build/test of gcc under cygwin and
>> reported the results to gcc-testresults. There may be is discussion
>> on the gcc mailing lists from c2000-2005. If you search
>> "site:gcc.gnu.org David Billinghurst cygwin" you ??might?? find
>> something relevant.
>> From memory, I got it all working by
>>  * building gcc and friends
>>  * using find to locate all the .exe and .dll files in the build
>>    tree
>>  * worked out by trial and error which files were needed at run time
>>    by the test suite.
>>  * setting PATH when running the testsuite so that the directories 
>>    containing (new) required .exe and .dll were in front of any
>>    system directories
>>  * making sure that PATH wasn't reset by the testsuite
>>  * looking at places where LD_LIBRARY_PATH was set/modified by the 
>>    testsuite and checking if cygwin needed PATH to match
>>  * (submitting patched to fix gcc testsuite under cygwin)
>> Once that was done it all "just worked" until it broke again. Good
>> luck.
> Thank you very much for this. This is the path that I was kind-of
> setting off on, I just wanted to try one more time to run the tests
> as-is, this time with only one make job and see if I could get the
> mass of failures to match so that I could say that my patches cause
> "no additional errors" (ignoring the fact that there's 16k total
> failures), but I can't even get the breakages to match up. (This is
> another topic, when I run tests in parallel I eventually end up with
> some "broken pipe" errors and that make job hangs).
> I found some of your patches, pity it got re-broken. I have a bug
> open for this here:
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=79867. In the end, I
> think that this should be fixed by adding some new "black box"
> interfaces to DejaGnu that manage the executable and library search
> paths. Then gcc's testsuite should deprecate ANY direct access to any
> of the *PATH environment variables in favor of this new interface in
> DejaGnu. As it is, the gcc code already changes both LD_LIBRARY_PATH
> and SHLIB_PATH to support HP-UX (not sure if any other systems use
> the latter), so it's already gotten a little hairy.
> In order to facilitate this cleanly, I think that libgcc needs to be
> moved out of /usr/bin and into /usr/lib/gcc/<triplet>/<version>/ and
> then have that added to the PATH and LD_LIBRARY_PATH somewhere
> (autoexec.bat? registry? I have no idea). Having an /sbin/ldconfig
> would be the most ideal mechanism of managing this. (Anybody want a
> little project? :) The test harness regularly toggles in between the
> host and target compiler.
> We need a clean and reproducible set of steps for running tests on
> Cygwin. Somewhere, these tests should be run regularly, maybe on a
> server/compiler farm somewhere under a VM, so that breakages can be
> addressed as soon as they appear rather than when somebody wants to
> touch the ms_abi code and has to test on Cygwin -- how I ended up
> here. :)

After any Windows Update, or a lot of package installs, you may want 
look at running
	rebase-trigger full[rebase] 
before rebooting to remove all Cygwin and Windows processes, then 
(with no Cygwin services or processes running) run Cygwin 
setup-x86{,_64} to rebase everything and maximize memory available 
to Cygwin processes.
This could be critical if you are doing any builds under Cygwin 32 
and have a lot of packages and/or large exes/dlls installed.

If you are running a lot of Cygwin services, cron or Scheduled Tasks, 
and/or background processes, you may want to look at running cygserver 
to cache process info and common system info (including SAM/AD).
Setup cygserver initially by running cygserver-config with 
elevated/admin rights, and start it at system startup by running 
cygrunsrv with elevated/admin rights.
If you ever expect to be running more than 62 simultaneous Cygwin 
processes on a system, bump kern.srv.process_cache_size in 
/etc/cygserver.conf created by cygserver-config to one of the higher 
values recommended in the man page.
I found this seemed to reduce process startup overhead and eliminated 
random broken pipes, hung, and failed processes due to a lot of shell 
forking.

Preallocate contiguous paging space at double RAM to reduce the chance 
that any set of Windows processes will reduce Windows free memory too 
low for an instant, and cause something to hang or fail, by giving 
Windows somewhere to page out a lot of LRU pages from inactive 
processes.
I found that if Windows (at least up to W7) free memory ever got too 
low, especially when all processors are pegged, it seemed unable to 
dynamically allocate and use more paging space to alleviate the 
crunch.
The situation may have improved on Windows 10, but I've already 
allocated the paging space, and uninstalled some (probably buggy) 
hogs that seemed to only ever allocate memory and never free any.

YMMV

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple