This is the mail archive of the systemtap@sourceware.org mailing list for the systemtap project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

proc memory statistics tapset


Hi,

I recently added printing of memory usage to each stap pass in verbose
mode to see how much (extra) memory each pass contributed. Which helped
me to keep track of memory usage while I refactored some of the dwfl
construction code.

It occurred to me this is actually what we need to use systemtap for. We
already have markers in stap itself that can be probed for the start and
end of each pass. So instead of adding new source code I should have
added a tapset that provides that information. That way it could be
reused by anybody that wants to trace a program and get memory
measurements/statistics. So that is what I just did.

With this new tapset you get the same results as stap -v (actually a bit
more and nicer human readable memory strings) with this oneliner: 

$ stap -e 'probe process("stap").mark("pass[0-3]*") { log($$name . "\t" . proc_mem_string()) }' -c 'stap -k -p4 testsuite/buildok/context_test.stp'
pass0__start	size: 58m, rss: 2528k, shr: 2068k, txt: 1352k, data: 428k
pass0__end	size: 58m, rss: 2548k, shr: 2088k, txt: 1352k, data: 428k
pass1a__start	size: 58m, rss: 2548k, shr: 2088k, txt: 1352k, data: 428k
pass1b__start	size: 58m, rss: 2748k, shr: 2240k, txt: 1352k, data: 428k
pass1__end	size: 74m, rss: 18m, shr: 2400k, txt: 1352k, data: 16m
pass2__start	size: 74m, rss: 18m, shr: 2400k, txt: 1352k, data: 16m
pass2__end	size: 74m, rss: 19m, shr: 2584k, txt: 1352k, data: 17m
pass3__start	size: 74m, rss: 19m, shr: 2584k, txt: 1352k, data: 17m
pass3__end	size: 74m, rss: 19m, shr: 2784k, txt: 1352k, data: 17m

But it is usable for any probe in any program (or the kernel, as long as
there is an associated current task for the probe point), and you don't
have to print anything, you can also query the different memory stats
individually and put them in an aggregate to show max/min/avg over time,
etc.

I would like to add it as tapset/proc_mem.stp and put the documentation
together with the Memory Tapset in the Tapset Reference manual since I
think they are generally useful. But if people feel they should go into
the examples only, that is fine too.

They are currently all marked /* unprivileged */ since the same
information is world readable through ps, top or /proc/pid/statm. But
the functions could also check is_myproc() and return zero in such
cases.

The code is slightly paranoid when it comes to fetching the mm struct,
but that was the only way I could see that made it safe. It only works
for the current task because that is what we can access lock free. For
any other task we would need to somehow keep some shadow bookkeeping to
prevent having to take locks on task and/or mm structs and I don't think
that is really worth it. The interesting memory stats are probably those
of the current process anyway.

Tested against 2.6.18 and 2.6.31.1 on x86_64 and 2.6.30 on i686, testson
other kernels and architectures or any other feedback very welcome.

Cheers,

Mark
// Process memory query and utility functions.
// Copyright (C) 2009 Red Hat Inc.
//
// This file is part of systemtap, and is free software.  You can
// redistribute it and/or modify it under the terms of the GNU General
// Public License (GPL); either version 2, or (at your option) any
// later version.

// <tapsetdescription>
// Process memory query and utility functions provide information about
// the memory usage of the current application. These functions provide
// information about the full size, resident, shared, code and data used
// by the current process. And provide utility functions to query the
// page size of the current architecture and create human readable string
// representations of bytes and pages used.
// </tapsetdescription>

%{
/* PF_BORROWED_MM got renamed to PF_KTHREAD with same semantics somewhere. */
#ifdef PF_BORROWED_MM
#define _STP_PF_KTHREAD PF_BORROWED_MM
#else
#define _STP_PF_KTHREAD PF_KTHREAD
#endif
  /* Returns the mm for the current proc. Slightly paranoid. Only returns
     from safe contexts (current must exist), and the task doesn't happens
     to be (coopted by) a kernel thread. Callers also check CONTEXT->regs. */
  static struct mm_struct *_stp_proc_mm(void)
  {
    struct task_struct *pid_task;
    struct mm_struct *mm;
    if (! current)
      return NULL;
    if (current->flags & _STP_PF_KTHREAD)
      return NULL;
    return current->mm;
  }
%}

/**
 * sfunction proc_mem_size - Total program virtual memory size in pages.
 *
 * Description: Returns the total virtual memory size in pages of the
 * current process, or zero when there is no current process or the
 * number of pages couldn't be retrieved.
 */
function proc_mem_size:long ()
%{ /* pure */ /* unprivileged */
   struct mm_struct *mm = _stp_proc_mm ();
   if (CONTEXT->regs && mm)
     THIS->__retvalue = mm->total_vm;
   else
     THIS->__retvalue = 0;
%}

/**
 * sfunction proc_mem_rss - Program resident set size in pages.
 *
 * Description: Returns the resident set size in pages of the current
 * process, or zero when there is no current process or the number of
 * pages couldn't be retrieved.
 */
function proc_mem_rss:long ()
%{ /* pure */ /* unprivileged */
   struct mm_struct *mm = _stp_proc_mm ();
   if (CONTEXT->regs && mm)
     THIS->__retvalue = (get_mm_counter(mm, file_rss)
                         + get_mm_counter(mm, anon_rss));
   else
     THIS->__retvalue = 0;  
%}

/**
 * sfunction proc_mem_shr - Program shared pages (from shared mappings).
 *
 * Description: Returns the shared pages (from shared mappings) of the
 * current process, or zero when there is no current process or the
 * number of pages couldn't be retrieved.
 */
function proc_mem_shr:long ()
%{ /* pure */ /* unprivileged */
   struct mm_struct *mm = _stp_proc_mm ();
   if (CONTEXT->regs && mm)
     THIS->__retvalue = get_mm_counter(mm, file_rss);
   else
     THIS->__retvalue = 0;  
%}

/**
 * sfunction proc_mem_txt - Program text (code) size in pages.
 *
 * Description: Returns the current process text (code) size in pages,
 * or zero when there is no current process or the number of pages
 * couldn't be retrieved.
 */
function proc_mem_txt:long ()
%{ /* pure */ /* unprivileged */
   struct mm_struct *mm = _stp_proc_mm ();
   if (CONTEXT->regs && mm)
     THIS->__retvalue = (PAGE_ALIGN(mm->end_code)
                         - (mm->start_code & PAGE_MASK)) >> PAGE_SHIFT;
   else
     THIS->__retvalue = 0;  
%}

/**
 * sfunction proc_mem_data - Program data size (data + stack) in pages.
 *
 * Description: Returns the current process data size (data + stack)
 * in pages, or zero when there is no current process or the number of
 * pages couldn't be retrieved.
 */
function proc_mem_data:long ()
%{ /* pure */ /* unprivileged */
   struct mm_struct *mm = _stp_proc_mm ();
   if (CONTEXT->regs && mm)
     THIS->__retvalue = mm->total_vm - mm->shared_vm;
   else
     THIS->__retvalue = 0;  
%}

/**
 * sfunction mem_page_size - Number of bytes in a page for this architecture.
 */
function mem_page_size:long ()
%{ /* pure */ /* unprivileged */
   THIS->__retvalue = PAGE_SIZE;
%}

/**
 * sfunction bytes_to_string - Human readable string for given bytes.
 *
 * Description: Returns a string representing the number of bytes
 * (when less than 5120b) postfixed by 'b', the number of kilobytes
 * (when less than 5120k) postfixed by 'k', the number of megabytes
 * (when less than 5120m) postfixed by 'm' or the number of gigabytes
 * postfixed by 'g'.
 */
function bytes_to_string:string (bytes:long)
{
  if (bytes < 5120)
    return sprintf("%db", bytes);
  bytes = bytes / 1024;
  if (bytes < 5120)
    return sprintf("%dk", bytes);
  bytes = bytes / 1024;
  if (bytes < 5120)
    return sprintf("%dm", bytes);
  bytes = bytes / 1024;
  return sprintf("%dg", bytes);
}

/**
 * sfunction pages_to_string - Turns pages into a human readable string.
 *
 * Description: Multiplies pages by page_size() to get the number of
 * bytes and returns the result of bytes_to_string().
 */
function pages_to_string:string (pages:long)
{
  bytes = pages * mem_page_size();
  return bytes_to_string (bytes);
}

/**
 * sfunction proc_mem_string - Human readable string of current proc memory usage.
 *
 * Description: Returns a human readable string showing the size, rss,
 * shr, txt and data of the memory used by the current process.
 * For example "size: 301m, rss: 11m, shr: 8m, txt: 52k, data: 2248k".
 */
function proc_mem_string:string ()
{
  return sprintf ("size: %s, rss: %s, shr: %s, txt: %s, data: %s",
                  pages_to_string(proc_mem_size()),
                  pages_to_string(proc_mem_rss()),
                  pages_to_string(proc_mem_shr()),
                  pages_to_string(proc_mem_txt()),
                  pages_to_string(proc_mem_data()));
}

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]