This is the mail archive of the
systemtap@sources.redhat.com
mailing list for the systemtap project.
Re: what if I wanted to write some tests...
Vara Prasad wrote:
Looking at this experience of Brad i am thinking (a loud) may be it is
time for us to do regular integration testing .
A simple system using the test bed to do a periodic build on UP/SMP
x86/x86_64 platforms for now, from the CVS and then probably run few
basic tests to see if they pass. Once these two steps are passed, create
a tar ball of the extracted code and have an automatic link to that tar
ball on the website.
This brings up following issues that we have to address
1) How often we check out the code and do the build?
My guess, considering we are not doing way too many changes at this time
probably once a week may be on Tuesday so that before Thursday we will
have issues observed to discuss on the call if need be. Of course if
someone checks a major change it would be nice to initiate this process
on demand as well.
Right now we don't have automatic testing of the build.
2) Where is the test bed?
I thought Will is looking into this using the RH test grid (Will correct
me if i am wrong). If this is correct is that the right test bed to do
this kind of build and sanity test? If this is not which other place we
can use? The other one that comes to my mind is the test suit Rusty
mentioned that he is planning to put together. Is there a test bed in
Intel that can pull the code and push the tar ball after successful
tests? Will there be any firewall issues in accessing and pushing the
tar ball to the website/CVS?
I have looked at the new and improve Red Hat test system. For the most
part we and to put the tests in the the "upstream" systemtap repository
and pull those tests into the testing system. The model used for things
like gcc. There can be additional one-off tests to make sure that a
particular bugzilla entry is addressed.
The tests are expected to run on RPM installed things. It is oriented to
install distro and run test in pristine environment. Until we have rpms
of systemtap the Red hat test system is not going to be testing things.
It would be good to set up minimal RPM packaging for systemtap.
Tests can be triggered to run when new versions of the package RPMs
become available. That should help with point 1 above. However, that is
dependent on new RPMs being generated. With packages developers don't
start a build every time a change is checked in
There are some mechanisms to allow partners to run tests outside. Tom
Kincaid at Red Hat should be able to provide more detail on how to
implement that.
3) How about other architectures mainly PPC 64 and IA 64?
Regarding PPC 64 i am thinking Ananth has access to PPC 64 machines that
he can access without firewall issues, so that might work . I don't know
if there are any IA64 machines available within the RH firewall if not
how Intel can can help here.
There should be test machines for the various architectures (x86,
x86-64, ppc64, and ia64) in the test infrastructure. However, things are
oriented to testing distributions, so this might not be as useful as
desired if the patches/changes are in someone local directory or source
repository rather than an RPM.
4) Do we have a sanity test suit for us to declare a good build?
For now i am thinking the examples in the CVS could be a good start. If
we have the kprobes tests Rusty/Anil was putting together that would be
good too.
We definitely want the kprobes testing. I took a look at the ia64 test.
Putting the kprobes at various points to make sure that particular
instructions are handled correctly by the probe is a good idea. However,
I found writing similar tests for x86 and x86-64 painful because of the
variable instruction size and having to make sure that things in three
different files agreed any time I made a change.
Any other issues i am missing?
Please let me know your thoughts on this proposal.
-Will