This is the mail archive of the gdb-patches@sourceware.org mailing list for the GDB project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: [RFC] Monster testcase generator for performance testsuite


Doug Evans <xdje42@gmail.com> writes:

Doug,
First of all, it is great to have such generator for performance testing,
but it doesn't have to be a monster and we don't need parallel build so
far.  The parallel build will get the generator over-complicated.  See
more below:

> This patch adds preliminary support for generating large programs.
> "Large" as in 10000 compunits or 5000 shared libraries or 3M ELF symbols.
>

Is there any reason we define the workload like this?  Can they
represent the typical and practical super large program?  I feel that the
workload you defined is too heavy to be practical, and the overweight
causes the long compilation time you mentioned below.

> There's still a bit more I want to add to this, but it's at a point
> where I can use it, and thus now's a good time to get some feedback.
>
> One difference between these tests and current perf tests is that
> one .exp is used to build the program and another .exp is used to
> run the test.  These programs take awhile to compile and link.
> Generating the sources for these monster testcases takes hardly any time
> at all relative to the amount of time to compile them.  I measured 13.5
> minutes to compile the included gmonster1 benchmark (with -j6!), and about
> an equivalent amount of time to run the benchmark.  Therefore it makes
> sense to be able to use one program in multiple performance tests, and
> therefore it makes sense to separate the build from the test run.

Compilation and run takes about 10 minutes respectively.  However, I
don't understand the importance that making tests running for 10
minutes, which is too long for a perf test case.  IMO, a-two-minute-run
program should be representative enough...

>
> These tests currently require separate build-perf and check-perf steps,
> which is different from normal perf tests.  However, due to the time
> it takes to build the program I've added support for building the pieces
> of the test in parallel, and hooking this parallel build support into
> the existing framework required some pragmatic compromise.

... so the parallel build part may not be needed.

> Running the gmonster1-ptype benchmark requires about 8G to link the program,
> and 11G to run it under gdb.  I still need to add the ability to
> have a small version enabled by default, and turn on the bigger version
> from the command line.  I don't expect everyone to have a big enough
> machine to run the test configuration that I do.

It looks like a monster rather than a perf test case :)  It is good to
have a small version enabled by default, which requires less than 1 G,
for example, to run it under GDB.  How much time it takes to compile
(sequential build) and run the small version?

-- 
Yao (éå)


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]