This is the mail archive of the
systemtap@sourceware.org
mailing list for the systemtap project.
Re: Labrat reports for 2.6.27-RC snapshot tests
- From: Kris Van Hees <kris dot van dot hees at oracle dot com>
- To: Mark Wielaard <mjw at redhat dot com>
- Cc: wenji dot huang at oracle dot com, SystemTAP <systemtap at sources dot redhat dot com>, "ZANNONI,ELENA" <ELENA dot ZANNONI at oracle dot com>, "VAN HEES,KRIS" <kris dot van dot hees at oracle dot com>
- Date: Mon, 27 Oct 2008 13:01:40 -0400
- Subject: Re: Labrat reports for 2.6.27-RC snapshot tests
- References: <490172AA.2000605@oracle.com> <1224848066.3418.20.camel@dijkstra.wildebeest.org>
On Fri, Oct 24, 2008 at 01:34:26PM +0200, Mark Wielaard wrote:
<< snip>>
> There is some noise in the output because the script seems to not
> understand some of the error messages when things fail. e.g.
>
> FAIL -> N/A test - systemtap.samples/gtod.exp(gtod (0))
> [...]
> N/A -> PASS test - systemtap.samples/gtod.exp(gtod (300))
>
> It would be useful to keep either the kernel version or the systemtap
> snapshot version constant between runs. It looks like currently both are
> updated at the same time. Although it isn't completely clear, the
> version is always reported as 0.7.1, it would be good to mention the
> last git commit (as in stap -V).
This is actually not something that can easily be solved without hardcoding
test-specific logic to strip out dynamic content in these messages. Since
labrat is not a project-specific reporting system, implementing such hardcoded
rules would be less desirable. Also note that wiring this logic in a way
that is not test specific is somewhat dangerous because you may end stripping
out important information from the messages unless people are extremely
careful in how they write the pass/fail messages.
If there is a very strict convention (one that gets enforced) on how dynamic
data is included in these messages, I can definitely look at implementing the
logic to strip it out based on the established conventions.
Kris