This is the mail archive of the systemtap@sources.redhat.com mailing list for the systemtap project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: Help with netlink needed


On Wed, 2005-04-06 at 14:05 -0400, Frank Ch. Eigler wrote:

> > Also, the point of having those status messages in a separate
> > channel would be so that they would be received, displayed, and
> > acted on immediately.  For example, the user-space daemon would need
> > to recognize fatal errors and optionally unload the module.
> 
> Can you explain why immediacy, in a sense such as "jumping the queue
> of already enqueued healthy entries in the trace buffer", is
> necessary?  After all, if the user-level daemon was already closely
> monitoring one channel in some sense, it could do the same with both.

Trace buffers may be large and take many  minutes to fill and be sent to
the daemon. The daemon should just be dumping them directly into files. 

Let me explain more clearly how I see this working.

Generally things are written to a "print buffer" using the internal
functions _stp_print_xxx().

_stp_print ("Output is: ");
_stp_printf ("pid is %d ", current->pid);
_stp_printf ("name is %s", current->comm);

before the probe returns it must call _stp_print_flush().  This
timestamps the accumulated print buffer and sends it to relayfs.
When relayfs fills an internal buffer, the user-space daemon is notified
data is ready and reads a bug per-cpu chunk, which contains a line like:
[123456.000002] Output is: pid is 1234 name is bash

The user-daemon (stpd) saves this data to a file named something like
"stpd_cpu2".  When the user hits ^c, a timer expires, or the probe
module notifies stpd (through a netlink command channel) that it wants
to terminate, stpd does "system(rmmod)" then collects the last output
before exiting.

This much I have working.  The next part I haven't given much thought,
but I assume we then run a program to read in all the data files run
them through some kind of filter. Thoughts?

As an option, if we don't need bulk per-cpu data, we can put
#define STP_NETLINK_ONLY
at the top of the module and all output will go over a netlink channel.
(This is not the same channel that relayfs uses for commands). One
example of a probe where this is useful is "shellsnoop" which displays
in realtime what files are being opened, read, and written by shell
commands.

For the SystemTap language user, most of the above will be invisible.
There will be a buffer management function to set preferences between
netlink, relayfs, etc.


> 
> > [...]  We may also want to support custom channels.  The probes can
> > define a new channel and instruct the daemon to save data on that
> > channel to a file or start up a program and forward the data stream
> > to and from it.  This would allow your variable snapshots, as well
> > as custom interfaces for monitoring kernel internals.
> 
> Yeah, maybe this would be interesting.  If implemented by using
> distinct relayfs channels (whatever form they take at user level),
> then my concern about a separate health channel is made moot.

I think it is moot. The relayfs libraries already implements a netlink
control channel for situations like what I described.  I have extended
it for my use and it is working nicely.

Custom channels would be a better fit to use  netlink instead of
relayfs.  Netlink is lower bandwidth, but is a single network stream vs
relayfs, which mmaps huge chunks of data into NR_CPUS files. 

Martin



Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]