This is the mail archive of the systemtap@sources.redhat.com mailing list for the systemtap project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: what if I wanted to write some tests...


Vara Prasad wrote:
Looking at this experience of Brad i am thinking (a loud) may be it is time for us to do regular integration testing .

A simple system using the test bed to do a periodic build on UP/SMP x86/x86_64 platforms for now, from the CVS and then probably run few basic tests to see if they pass. Once these two steps are passed, create a tar ball of the extracted code and have an automatic link to that tar ball on the website.

This brings up following issues that we have to address

1) How often we check out the code and do the build?
My guess, considering we are not doing way too many changes at this time probably once a week may be on Tuesday so that before Thursday we will have issues observed to discuss on the call if need be. Of course if someone checks a major change it would be nice to initiate this process on demand as well.

Right now we don't have automatic testing of the build.


2) Where is the test bed?
I thought Will is looking into this using the RH test grid (Will correct me if i am wrong). If this is correct is that the right test bed to do this kind of build and sanity test? If this is not which other place we can use? The other one that comes to my mind is the test suit Rusty mentioned that he is planning to put together. Is there a test bed in Intel that can pull the code and push the tar ball after successful tests? Will there be any firewall issues in accessing and pushing the tar ball to the website/CVS?

I have looked at the new and improve Red Hat test system. For the most part we and to put the tests in the the "upstream" systemtap repository and pull those tests into the testing system. The model used for things like gcc. There can be additional one-off tests to make sure that a particular bugzilla entry is addressed.


The tests are expected to run on RPM installed things. It is oriented to install distro and run test in pristine environment. Until we have rpms of systemtap the Red hat test system is not going to be testing things. It would be good to set up minimal RPM packaging for systemtap.

Tests can be triggered to run when new versions of the package RPMs become available. That should help with point 1 above. However, that is dependent on new RPMs being generated. With packages developers don't start a build every time a change is checked in

There are some mechanisms to allow partners to run tests outside. Tom Kincaid at Red Hat should be able to provide more detail on how to implement that.

3) How about other architectures mainly PPC 64 and IA 64?

Regarding PPC 64 i am thinking Ananth has access to PPC 64 machines that he can access without firewall issues, so that might work . I don't know if there are any IA64 machines available within the RH firewall if not how Intel can can help here.

There should be test machines for the various architectures (x86, x86-64, ppc64, and ia64) in the test infrastructure. However, things are oriented to testing distributions, so this might not be as useful as desired if the patches/changes are in someone local directory or source repository rather than an RPM.


4) Do we have a sanity test suit for us to declare a good build?

For now i am thinking the examples in the CVS could be a good start. If we have the kprobes tests Rusty/Anil was putting together that would be good too.

We definitely want the kprobes testing. I took a look at the ia64 test. Putting the kprobes at various points to make sure that particular instructions are handled correctly by the probe is a good idea. However, I found writing similar tests for x86 and x86-64 painful because of the variable instruction size and having to make sure that things in three different files agreed any time I made a change.


Any other issues i am missing?

Please let me know your thoughts on this proposal.

-Will



Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]