This is the mail archive of the
gdb-patches@sourceware.org
mailing list for the GDB project.
RE: [RFC] GDB performance testing infrastructure
- From: "Agovic, Sanimir" <sanimir dot agovic at intel dot com>
- To: 'Yao Qi' <yao at codesourcery dot com>
- Cc: "gdb-patches at sourceware dot org" <gdb-patches at sourceware dot org>
- Date: Tue, 27 Aug 2013 13:49:23 +0000
- Subject: RE: [RFC] GDB performance testing infrastructure
- Authentication-results: sourceware.org; auth=none
- References: <520B7F70 dot 6070207 at codesourcery dot com>
Hello Yao,
I like the overall proposal for a "micro" benchmark suite. Some comments below.
> -----Original Message-----
> From: gdb-patches-owner@sourceware.org [mailto:gdb-patches-owner@sourceware.org] On Behalf
> Of Yao Qi
> Sent: Wednesday, August 14, 2013 03:01 PM
> To: gdb-patches@sourceware.org
> Subject: [RFC] GDB performance testing infrastructure
>
> * Remote debugging. It is slower to read from the remote target, and
> worse, GDB reads the same memory regions in multiple times, or reads
> the consecutive memory by multiple packets.
>
Once gdb and gdbserver share most of the target code, the overhead will be
caused by the serial protocol roundtrips. But this will take a while...
> * Tracepoint. Tracepoint is designed to be efficient on collecting
> data in the inferior, so we need performance tests to guarantee that
> tracepoint is still efficient enough. Note that we a test
> `gdb.trace/tspeed.exp', but there are still some rooms to improve.
>
Afaik the tracepoint functionality is quite separated from gdb may be tested
in isolation. Having a generic benchmark framework covering the most parts of
gdb is probably _the_ way to go but I see some room for specialized benchmarks
e.g. for tracepoints with a custom driver. But my knowledge is too vague on
the topic.
> 2. Detect performance regressions. We collected the performance data
> of each micro-benchmark, and we need to detect or identify the
> performance regression by comparing with the previous run. It is
> more powerful to associate it with continuous testing.
>
Something really simple, so simple that one could run it silently with every
make invokation. For a newcomer, it took me some time to get used to make
check e.g. setup, run, and interpret the tests with various settings. Something
simpler would help to run it more often.
>
> 2 Known works
> =============
>
> * [LNT] It was written for LLVM, but is *designed* to be usable for
> the performance testing of any software. It is written in python,
> well-documented and easy to set up. LNT spawn the compiler first
> and then target program, record the time usages of compiler and
> target program in json format. No interaction is involved. The
> performance data collection in LNT is relatively simple, because it
> is targeted to compiler. The performance testing part is done, and
> the next step is to show the data and detect performance
> regressions. LNT does a lot work here. The performance data in
> json format can be imported to a database, and shown through [web].
> The performance regression will be highlighted in red.
>
> * [lldb] LLDB has a [performance.py] to measure the speed and memory
> usage of LLDB. It captures the internal events, feeds some events
> and record the time usages. It handles interactions by consuming
> debugging events, and take some actions accordingly. It only
> collects performance data, doesn't detect performance regressions.
>
> * libstdc++-v3 There is directory performance in
> libstdc++-v3/testsuite/ and a header testsuite_performance.h in
> testsuite/util/. Test cases are compiled with the header, and run
> with some large data set, to calculate the time usage. It is
> suitable for performance testing for a library.
>
I like to add the Machine Interface (MI) to the list, but it is quite rudimentary:
$ gdb -interpreter mi -q debugee
[...]
-enable-timings
^done
(gdb)
-break-insert -f main
^done,bkpt={...},time={wallclock="0.00656",user="0.00000",system="0.00000"}
[...]
(gdb)
-exec-step
^running
*running,thread-id="1"
(gdb)
*stopped,[...],time={wallclock="0.19425",user="0.09700",system="0.04200"}
(gdb)
With -enable-timings[1] enabled, every result record has a time triple
appended, even for async[2] ones. If we come up with a full mi parser
one could run tests w/o timings. A mi result is quite json-ish.
(To be honest I do not know how timings are composed of =D)
In addition there are some tools for plotting benchmark results[3].
[1] http://sourceware.org/gdb/onlinedocs/gdb/GDB_002fMI-Miscellaneous-Commands.html
[2] https://sourceware.org/gdb/onlinedocs/gdb/GDB_002fMI-Async-Records.html
[3] http://speed.pypy.org/
-Sanimir
Intel GmbH
Dornacher Strasse 1
85622 Feldkirchen/Muenchen, Deutschland
Sitz der Gesellschaft: Feldkirchen bei Muenchen
Geschaeftsfuehrer: Christian Lamprechter, Hannes Schwaderer, Douglas Lusk
Registergericht: Muenchen HRB 47456
Ust.-IdNr./VAT Registration No.: DE129385895
Citibank Frankfurt a.M. (BLZ 502 109 00) 600119052