Re: [PATCH] Add support for multiple data points to perf tests

[Doug Evans <xdje42 at gmail dot com>]

On Mon, Jul 27, 2015 at 7:20 AM, Yao Qi <qiyaoltc@gmail.com> wrote:
> Doug Evans <xdje42@gmail.com> writes:
>
>> This patch does two things.
>> 1) Add support for multiple data points.
>> 2) Move the "report" output from perftest.log to perftest.sum.
>>
>
> What does perftest.sum look like if we write test results by a different
> reporter? say xml reporter or jason reporter?

Hi.
This question feels misguided, perhaps I haven't
sufficiently explained what I'm doing.
These same questions can be asked of perftest.log in the current tree.
All I've done is rename perftest.log to perftest.sum,
and then put more test results in text form in perftest.log.
I *could* put it all in one file, but I like what's in perftest.log today,
(though there are a few tweaks I'd like to make),
I'm just moving it to perftest.sum.
Following the pattern of file naming already in use
will be helpful to everyone using the perf testsuite.

If/when we have other forms of output I would expect them
to not go to .log or .sum files.

>> I want to record the raw data somewhere, and a bit of statistical analysis
>> (standard deviation left for another day), but I also don't want
>> it to clutter up the basic report.
>> This patch takes a cue from gdb.{sum,log} and does the same thing
>> with perftest.{sum,log}.
>
> I'd like pertest framework only writes raw data down in some formats
> (such as text, xml or jason) without any analysis.  Statistical analysis
> can be performed in a post-mortem way by other scripts.

We can generate a .csv file with the raw data if/when the need arises.
[or whatever is easiest for post-processing scripts to read]

A human readable form with basic statistical analysis
is useful today, and doable in 5 lines of code.
Let's add the complexity when there's a demonstrated
need for it.