RE: Tests that require 'modprobe <module>'

["Stone, Joshua I" <joshua dot i dot stone at intel dot com>]

Mike Mason wrote:
>  I understand the desire to reduce the root requirement for users,
> but for complete testing that may be unrealistic.

It's not unrealistic if you make it a goal.  Grep'ing for sudo in the
testsuite right now, I see only two ways that sudo is used:

systemtap.printf/*b.exp -- calls 'sudo rm' to cleanup temp files.  This
shouldn't be necessary, since the files are owned by the user running
the test.

systemtap.context/context.exp -- builds and loads a test kernel module.
Clever use of embedded-C in a stap-script could generate and load the
test modules without needing an explicit insmod.  There's still an issue
with resolving symbols for this test to work, but bug #2151 would help.

> If the code isn't in the kernel and isn't built as a module, the
> tests for those probe points should fail.  I know, the test didn't
> actually run and fail, we just failed to test.  To me, that's still
> a failure.

But it's a failure of a different sort.  That's what skipped &
unsupported result codes are for.

> As for whether the module's in use, in some cases that doesn't matter.
> For example, you can still load the nfs, nfsd and sunrpc modules and
> run build tests against them even though they're not being used.  This
> may not be true for all modules.

I suppose I can accept that, as long as you can restore the system when
you're done.  IIRC, some modules may trigger loading other modules
though, so cleanup may be difficult.

>>  Can you instead write your test to gracefully skip if the module
>> isn't present?  (It shouldn't record a failure...)
>  I don't think untested tapsets should be silently ignored.  If we
> don't figure out how to test *everything*, even if it's hard, how
> will we know when some change in the kernel breaks a tapset?  When a
> systemtap user stumbles on the problem?  IMHO, that's unacceptable.

I don't mean to call it a 'pass' -- again, that's what skipped &
unsupported are for.  It's important to distinguish between "I can't
test that on this system" and "something is really broken".

Josh