This is the mail archive of the libc-alpha@sourceware.org mailing list for the glibc project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: [PATCH] Optimize strstr, strcasestr and memmem


On Mon, May 21, 2012 at 2:18 PM, Maxim Kuvyrkov <maxim@codesourcery.com> wrote:
>> This should not preclude other people from contributing machine
>> specific benchmarks.
>
> Certainly. ?The design and thought process behind it that I outlined will address the most immediate need for creating at least /some/ performance baseline for GLIBC.

Sounds good.

>>> The benchmark design goals that I have so far:

Could you please start a wiki page for this work if you haven't
already? That would help me point other people at the design and
goals.

>> Why "fixed time?" Why not "fixed number of iterations?" It is already
>> difficult for maintainers to evaluate TIMEOUT_FACTOR or TIMEOUT in the
>> tests, extending this to the benchmarks is like throwing salt on a
>> wound. I would prefer to see a fixed number of test executions rather
>> than a fixed amount of time.
>
> Why would we want to evaluate TIMEOUT_FACTOR and TIMEOUT for benchmarks beyond setting it to a single value for all tests? ?The mechanism for controlling benchmark runtime that I have in mind is that the benchmark body is run in "while (run_benchmark) {}" loop with an alarm set to BENCHMARK_TIME seconds (30-120 seconds). ?When the alarm goes off the signal handler sets "run_benchmark = false" and stops the execution of the benchmark, but allowing it to finish the current iteration.
>
> For most benchmarks one can get reasonably-precise results from a 30-120 second run. ?Runs below 10 seconds will have too much of startup/warmup error. ?Runs above 120 seconds will be just wasting CPU time (assuming that the benchmark body can execute within 1-5 seconds, so that we get well-averaged results).
>
> I'm tired of hand-tweaking iteration numbers for all the different systems out there.

What do you do if you exceed the runtime but are in the middle of an interation?

>> Do you have a plan for the tests that don't run via test-skeleton.c? I
>> would love to see test-skeleton.c expanded and converted to support
>> all the tests.
>
> The plan is to not handle them until they are converted to use test-skeleton.c.

OK, please state this in your design wiki page.

>> We need to collect data, process it, and make some graphs, and keep
>> doing that over and over.
>>
>> We could easily add another git repo of just performance data?
>>
>> That way the CI system checks out the performance data, appends to it
>> after a run, and pushes the performance data out.
>>
>> Then you could have consumers pull the new data and update
>> visualizations based on that data?
>
> This is a great idea. ?I would love to see a [semi-]public-writable git repo where people could upload their benchmark results with a simple "make upload-benchmark" rule. ?Kind of next level up from testresult mailing lists that many projects have. ?With a file-per-machine policy one can then easily generate plottable format from the git history of the file.

Agreed.

Would you add this to your design so we have a coherent whole for
people to refer to?

Cheers,
Carlos.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]