This is the mail archive of the
gdb-patches@sourceware.org
mailing list for the GDB project.
Re: [RFC] Monster testcase generator for performance testsuite
- From: Yao Qi <yao at codesourcery dot com>
- To: Doug Evans <xdje42 at gmail dot com>
- Cc: <gdb-patches at sourceware dot org>
- Date: Mon, 5 Jan 2015 21:32:31 +0800
- Subject: Re: [RFC] Monster testcase generator for performance testsuite
- Authentication-results: sourceware.org; auth=none
- References: <m3lhllpkd6 dot fsf at seba dot sebabeach dot org>
Doug Evans <xdje42@gmail.com> writes:
Doug,
First of all, it is great to have such generator for performance testing,
but it doesn't have to be a monster and we don't need parallel build so
far. The parallel build will get the generator over-complicated. See
more below:
> This patch adds preliminary support for generating large programs.
> "Large" as in 10000 compunits or 5000 shared libraries or 3M ELF symbols.
>
Is there any reason we define the workload like this? Can they
represent the typical and practical super large program? I feel that the
workload you defined is too heavy to be practical, and the overweight
causes the long compilation time you mentioned below.
> There's still a bit more I want to add to this, but it's at a point
> where I can use it, and thus now's a good time to get some feedback.
>
> One difference between these tests and current perf tests is that
> one .exp is used to build the program and another .exp is used to
> run the test. These programs take awhile to compile and link.
> Generating the sources for these monster testcases takes hardly any time
> at all relative to the amount of time to compile them. I measured 13.5
> minutes to compile the included gmonster1 benchmark (with -j6!), and about
> an equivalent amount of time to run the benchmark. Therefore it makes
> sense to be able to use one program in multiple performance tests, and
> therefore it makes sense to separate the build from the test run.
Compilation and run takes about 10 minutes respectively. However, I
don't understand the importance that making tests running for 10
minutes, which is too long for a perf test case. IMO, a-two-minute-run
program should be representative enough...
>
> These tests currently require separate build-perf and check-perf steps,
> which is different from normal perf tests. However, due to the time
> it takes to build the program I've added support for building the pieces
> of the test in parallel, and hooking this parallel build support into
> the existing framework required some pragmatic compromise.
... so the parallel build part may not be needed.
> Running the gmonster1-ptype benchmark requires about 8G to link the program,
> and 11G to run it under gdb. I still need to add the ability to
> have a small version enabled by default, and turn on the bigger version
> from the command line. I don't expect everyone to have a big enough
> machine to run the test configuration that I do.
It looks like a monster rather than a perf test case :) It is good to
have a small version enabled by default, which requires less than 1 G,
for example, to run it under GDB. How much time it takes to compile
(sequential build) and run the small version?
--
Yao (éå)