RFA: handle "MiniDebuginfo" section

[Tom Tromey <tromey at redhat dot com>]

Fedora recently approved and committed the "MiniDebuginfo" feature:

    http://fedoraproject.org/wiki/Features/MiniDebugInfo

Essentially this is a way to ship some extra symbols, compressed in a
special section in an executable or library, so that backtracing can
work "well enough" without requiring full debuginfo.

This patch adds support for this feature to gdb.  I think the
implementation is pretty straightforward.  If you want to enable it, you
will need the LZMA library:

    http://tukaani.org/lzma/

Built and regtested on x86-64 Fedora 16.

At least a doc review is required.

Tom

2012-11-09  Alexander Larsson  <alexl@redhat.com>
	    Jan Kratochvil  <jan.kratochvil@redhat.com>

	* elfread.c (alloc_lzma, free_lzma): New functions.
	(gdb_lzma_allocator): New global.
	(struct lzma_stream): New.
	(lzma_open, lzma_pread, lzma_close, lzma_stat)
	(find_separate_debug_file_in_section): New functions.
	(elf_symfile_read): Call find_separate_debug_file_in_section if no
	other debuginfo is found.
	* configure.ac: Check for lzma.
	* configure, config.in: Rebuild.
	* Makefile.in (LIBLZMA): New variable.
	(CLIBS): Include LIBLZMA.
	* NEWS: Mention mini debuginfo feature.

2012-11-09  Tom Tromey  <tromey@redhat.com>

	* gdb.texinfo (Separate Debug Section): New node.
	(GDB Files): Update.

2012-11-09  Jan Kratochvil  <jan.kratochvil@redhat.com>

	* gdb.dwarf2/dw2-gnu-debugdata.exp: New file.
	* gdb.dwarf2/dw2-gnu-debugdata.c: New file.

diff --git a/gdb/Makefile.in b/gdb/Makefile.in
index 9e7702d..a78917e 100644
--- a/gdb/Makefile.in
+++ b/gdb/Makefile.in
@@ -151,6 +151,9 @@ READLINE_CFLAGS = @READLINE_CFLAGS@
 # Where is expat?  This will be empty if expat was not available.
 LIBEXPAT = @LIBEXPAT@
 
+# Where is lzma?  This will be empty if lzma was not available.
+LIBLZMA = @LIBLZMA@
+
 WARN_CFLAGS = @WARN_CFLAGS@
 WERROR_CFLAGS = @WERROR_CFLAGS@
 GDB_WARN_CFLAGS = $(WARN_CFLAGS)
@@ -469,7 +472,7 @@ INTERNAL_LDFLAGS = $(CFLAGS) $(GLOBAL_CFLAGS) $(MH_LDFLAGS) $(LDFLAGS) $(CONFIG_
 # LIBIBERTY appears twice on purpose.
 CLIBS = $(SIM) $(READLINE) $(OPCODES) $(BFD) $(INTL) $(LIBIBERTY) $(LIBDECNUMBER) \
 	$(XM_CLIBS) $(NAT_CLIBS) $(GDBTKLIBS) @LIBS@ @PYTHON_LIBS@ \
-	$(LIBEXPAT) \
+	$(LIBEXPAT) $(LIBLZMA) \
 	$(LIBIBERTY) $(WIN32LIBS) $(LIBGNU)
 CDEPS = $(XM_CDEPS) $(NAT_CDEPS) $(SIM) $(BFD) $(READLINE_DEPS) \
 	$(OPCODES) $(INTL_DEPS) $(LIBIBERTY) $(CONFIG_DEPS) $(LIBGNU)
diff --git a/gdb/NEWS b/gdb/NEWS
index 8567742..2102d90 100644
--- a/gdb/NEWS
+++ b/gdb/NEWS
@@ -67,6 +67,11 @@ py [command]
   ** Memory changes are now notified using new async record
      "=memory-changed".
 
+* GDB now supports the "mini debuginfo" section, .gnu_debugdata.
+  You must have the LZMA library available when configuring GDB for this
+  feature to be enabled.  For more information, see:
+      http://fedoraproject.org/wiki/Features/MiniDebugInfo
+
 *** Changes in GDB 7.5
 
 * GDB now supports x32 ABI.  Visit <http://sites.google.com/site/x32abi/>
diff --git a/gdb/configure.ac b/gdb/configure.ac
index f0b7df3..c751c2d 100644
--- a/gdb/configure.ac
+++ b/gdb/configure.ac
@@ -2056,6 +2056,27 @@ LIBS=$OLD_LIBS
 # Add any host-specific objects to GDB.
 CONFIG_OBS="${CONFIG_OBS} ${gdb_host_obs}"
 
+# If building on ELF, look for lzma support for embedded compressed debug info.
+if test $gdb_cv_var_elf = yes; then
+  AC_ARG_WITH(lzma,
+    AS_HELP_STRING([--with-lzma], [support lzma compression (auto/yes/no)]),
+    [], [with_lzma=auto])
+  AC_MSG_CHECKING([whether to use lzma])
+  AC_MSG_RESULT([$with_lzma])
+
+  if test "${with_lzma}" != no; then
+    AC_LIB_HAVE_LINKFLAGS([lzma], [], [#include "lzma.h"],
+			  [lzma_index_iter iter;
+			   lzma_index_iter_init (&iter, 0);
+			   lzma_mf_is_supported (LZMA_MF_HC3);])
+    if test "$HAVE_LIBLZMA" != yes; then
+      if test "$with_lzma" = yes; then
+        AC_MSG_ERROR([missing liblzma for --with-lzma])
+      fi
+    fi
+  fi
+fi
+
 LIBGUI="../libgui/src/libgui.a"
 GUI_CFLAGS_X="-I${srcdir}/../libgui/src"
 AC_SUBST(LIBGUI)
diff --git a/gdb/doc/gdb.texinfo b/gdb/doc/gdb.texinfo
index 50fc123..8c85a19 100644
--- a/gdb/doc/gdb.texinfo
+++ b/gdb/doc/gdb.texinfo
@@ -15787,6 +15787,7 @@ program.  To debug a core dump of a previous run, you must also tell
 @menu
 * Files::                       Commands to specify files
 * Separate Debug Files::        Debugging information in separate files
+* Separate Debug Section::      Debugging information in a special section
 * Index Files::                 Index files speed up GDB
 * Symbol Errors::               Errors reading symbol files
 * Data Files::                  GDB data files
@@ -16712,6 +16713,44 @@ gnu_debuglink_crc32 (unsigned long crc,
 @noindent
 This computation does not apply to the ``build ID'' method.
 
+@node Separate Debug Section
+@section Debugging information in a special section
+@cindex separate debug sections
+@cindex @samp{.gnu_debugdata} section
+
+Some systems ship pre-built executables and libraries that have a
+special @samp{.gnu_debugdata} section.  This section holds an
+LZMA-compressed ELF object and is used to supply extra symbols for
+backtraces.  @value{GDBN} has support for this extension.
+
+This section can be easily created using @command{objcopy} and other
+standard utilities:
+
+@smallexample
+# Extract the dynamic symbols from the main binary, there is no need
+# to also have these in the normal symbol table
+nm -D @var{binary} --format=posix --defined-only \
+  | awk '@{ print $1 @}' | sort > dynsyms
+
+# Extract all the text (i.e. function) symbols from the debuginfo .
+nm @var{binary} --format=posix --defined-only \
+  | awk '@{ if ($2 == "T" || $2 == "t") print $1 @}' \
+  | sort > funcsyms
+
+# Keep all the function symbols not already in the dynamic symbol
+# table.
+comm -13 dynsyms funcsyms > keep_symbols
+
+# Copy the full debuginfo, keeping only a minimal set of symbols and
+# removing some unnecessary sections.
+objcopy -S --remove-section .gdb_index --remove-section .comment \
+  --keep-symbols=keep_symbols @var{binary} mini_debuginfo
+
+# Inject the compressed data into the .gnu_debugdata section of the
+# original binary.
+xz mini_debuginfo
+objcopy --add-section .gnu_debugdata=mini_debuginfo.xz @var{binary}
+@end smallexample
 
 @node Index Files
 @section Index Files Speed Up @value{GDBN}
diff --git a/gdb/elfread.c b/gdb/elfread.c
index 516cbd0..789b649 100644
--- a/gdb/elfread.c
+++ b/gdb/elfread.c
@@ -45,6 +45,10 @@
 #include "regcache.h"
 #include "bcache.h"
 #include "gdb_bfd.h"
+#include "gdbcore.h"
+#ifdef HAVE_LIBLZMA
+# include <lzma.h>
+#endif
 
 extern void _initialize_elfread (void);
 
@@ -1214,6 +1218,264 @@ find_separate_debug_file_by_buildid (struct objfile *objfile)
   return NULL;
 }
 
+#ifdef HAVE_LIBLZMA
+
+/* Custom lzma_allocator.alloc so they use the gdb ones.  */
+
+static void *
+alloc_lzma (void *opaque, size_t nmemb, size_t size)
+{
+  return xmalloc (nmemb * size);
+}
+
+/* Custom lzma_allocator.free so they use the gdb ones.  */
+
+static void
+free_lzma (void *opaque, void *ptr)
+{
+  xfree (ptr);
+}
+
+/* It cannot be const due to the lzma library function prototypes.  */
+
+static lzma_allocator gdb_lzma_allocator = { alloc_lzma, free_lzma, NULL};
+
+/* Custom bfd_openr_iovec implementation to read compressed data from a
+   section. This keeps only the last decompressed block in memory to
+   allow larger data without using to much memory.  */
+
+struct lzma_stream
+{
+  /* Section of input BFD we are decoding data from.  */
+  asection *section;
+
+  /* lzma library decompression state.  */
+  lzma_index *index;
+
+  /* Currently decoded block.  */
+  bfd_size_type data_start;
+  bfd_size_type data_end;
+  gdb_byte *data;
+};
+
+/* bfd_openr_iovec OPEN_P implementation for
+   find_separate_debug_file_in_section.  OPEN_CLOSURE is 'asection *' of the
+   section to decompress.
+
+   Return 'struct lzma_stream *' must be freed by caller by xfree, together
+   with its INDEX lzma data.  */
+
+static void *
+lzma_open (struct bfd *nbfd, void *open_closure)
+{
+  asection *section = open_closure;
+  bfd_size_type size, offset;
+  lzma_stream_flags options;
+  gdb_byte footer[LZMA_STREAM_HEADER_SIZE];
+  gdb_byte *indexdata;
+  lzma_index *index;
+  int ret;
+  uint64_t memlimit = UINT64_MAX;
+  struct lzma_stream *lstream;
+  size_t pos;
+
+  size = bfd_get_section_size (section);
+  offset = section->filepos + size - LZMA_STREAM_HEADER_SIZE;
+  if (size < LZMA_STREAM_HEADER_SIZE
+      || bfd_seek (section->owner, offset, SEEK_SET) != 0
+      || bfd_bread (footer, LZMA_STREAM_HEADER_SIZE, section->owner)
+         != LZMA_STREAM_HEADER_SIZE
+      || lzma_stream_footer_decode (&options, footer) != LZMA_OK
+      || offset < options.backward_size)
+    {
+      bfd_set_error (bfd_error_wrong_format);
+      return NULL;
+    }
+
+  offset -= options.backward_size;
+  indexdata = xmalloc (options.backward_size);
+  index = NULL;
+  pos = 0;
+  if (bfd_seek (section->owner, offset, SEEK_SET) != 0
+      || bfd_bread (indexdata, options.backward_size, section->owner)
+         != options.backward_size
+      || lzma_index_buffer_decode (&index, &memlimit, &gdb_lzma_allocator,
+				   indexdata, &pos, options.backward_size)
+         != LZMA_OK
+      || lzma_index_size (index) != options.backward_size)
+    {
+      xfree (indexdata);
+      bfd_set_error (bfd_error_wrong_format);
+      return NULL;
+    }
+  xfree (indexdata);
+
+  lstream = xzalloc (sizeof (struct lzma_stream));
+  lstream->section = section;
+  lstream->index = index;
+
+  return lstream;
+}
+
+/* bfd_openr_iovec PREAD_P implementation for
+   find_separate_debug_file_in_section.  Passed STREAM
+   is 'struct lzma_stream *'.  */
+
+static file_ptr
+lzma_pread (struct bfd *nbfd, void *stream, void *buf, file_ptr nbytes,
+	    file_ptr offset)
+{
+  struct lzma_stream *lstream = stream;
+  bfd_size_type chunk_size;
+  lzma_index_iter iter;
+  gdb_byte *compressed, *uncompressed;
+  file_ptr block_offset;
+  lzma_filter filters[LZMA_FILTERS_MAX + 1];
+  lzma_block block;
+  size_t compressed_pos, uncompressed_pos;
+  file_ptr res;
+
+  res = 0;
+  while (nbytes > 0)
+    {
+      if (lstream->data == NULL
+	  || lstream->data_start > offset || offset >= lstream->data_end)
+	{
+	  asection *section = lstream->section;
+
+	  lzma_index_iter_init (&iter, lstream->index);
+	  if (lzma_index_iter_locate (&iter, offset))
+	    break;
+
+	  compressed = xmalloc (iter.block.total_size);
+	  block_offset = section->filepos + iter.block.compressed_file_offset;
+	  if (bfd_seek (section->owner, block_offset, SEEK_SET) != 0
+	      || bfd_bread (compressed, iter.block.total_size, section->owner)
+		 != iter.block.total_size)
+	    {
+	      xfree (compressed);
+	      break;
+	    }
+
+	  uncompressed = xmalloc (iter.block.uncompressed_size);
+
+	  memset (&block, 0, sizeof (block));
+	  block.filters = filters;
+	  block.header_size = lzma_block_header_size_decode (compressed[0]);
+	  if (lzma_block_header_decode (&block, &gdb_lzma_allocator, compressed)
+	      != LZMA_OK)
+	    {
+	      xfree (compressed);
+	      xfree (uncompressed);
+	      break;
+	    }
+
+	  compressed_pos = block.header_size;
+	  uncompressed_pos = 0;
+	  if (lzma_block_buffer_decode (&block, &gdb_lzma_allocator,
+					compressed, &compressed_pos,
+					iter.block.total_size,
+					uncompressed, &uncompressed_pos,
+					iter.block.uncompressed_size)
+	      != LZMA_OK)
+	    {
+	      xfree (compressed);
+	      xfree (uncompressed);
+	      break;
+	    }
+
+	  xfree (compressed);
+
+	  xfree (lstream->data);
+	  lstream->data = uncompressed;
+	  lstream->data_start = iter.block.uncompressed_file_offset;
+	  lstream->data_end = (iter.block.uncompressed_file_offset
+			       + iter.block.uncompressed_size);
+	}
+
+      chunk_size = min (nbytes, lstream->data_end - offset);
+      memcpy (buf, lstream->data + offset - lstream->data_start, chunk_size);
+      buf = (gdb_byte *) buf + chunk_size;
+      offset += chunk_size;
+      nbytes -= chunk_size;
+      res += chunk_size;
+    }
+
+  return res;
+}
+
+/* bfd_openr_iovec CLOSE_P implementation for
+   find_separate_debug_file_in_section.  Passed STREAM
+   is 'struct lzma_stream *'.  */
+
+static int
+lzma_close (struct bfd *nbfd,
+	    void *stream)
+{
+  struct lzma_stream *lstream = stream;
+
+  lzma_index_end (lstream->index, &gdb_lzma_allocator);
+  xfree (lstream->data);
+  xfree (lstream);
+  return 0;
+}
+
+/* bfd_openr_iovec STAT_P implementation for
+   find_separate_debug_file_in_section.  Passed STREAM
+   is 'struct lzma_stream *'.  */
+
+static int
+lzma_stat (struct bfd *abfd,
+	   void *stream,
+	   struct stat *sb)
+{
+  struct lzma_stream *lstream = stream;
+
+  sb->st_size = lzma_index_uncompressed_size (lstream->index);
+  return 0;
+}
+
+/* This looks for a xz compressed separate debug info object file embedded
+   in a section called .gnu_debugdata.  See
+   http://fedoraproject.org/wiki/Features/MiniDebugInfo
+   or the "Separate Debug Sections" of the manual for details.
+   If we find one we create a iovec based bfd that decompresses the
+   object data on demand.  If we don't find one, return NULL.  */
+
+static bfd *
+find_separate_debug_file_in_section (struct objfile *objfile)
+{
+  asection *section;
+  bfd *abfd;
+
+  section = bfd_get_section_by_name (objfile->obfd, ".gnu_debugdata");
+  if (section == NULL)
+    return NULL;
+
+  abfd = gdb_bfd_openr_iovec (objfile->name, gnutarget, lzma_open, section,
+			      lzma_pread, lzma_close, lzma_stat);
+  if (abfd == NULL)
+    return NULL;
+
+  if (!bfd_check_format (abfd, bfd_object))
+    {
+      gdb_bfd_unref (abfd);
+      return NULL;
+    }
+
+  return abfd;
+}
+
+#else /* !HAVE_LIBLZMA */
+
+static bfd *
+find_separate_debug_file_in_section (struct objfile *objfile)
+{
+  return NULL;
+}
+
+#endif /* !HAVE_LIBLZMA */
+
 /* Scan and build partial symbols for a symbol file.
    We have been initialized by a call to elf_symfile_init, which
    currently does nothing.
@@ -1437,21 +1699,31 @@ elf_symfile_read (struct objfile *objfile, int symfile_flags)
   else if (!objfile_has_partial_symbols (objfile))
     {
       char *debugfile;
+      bfd *abfd = NULL;
+      struct cleanup *cleanup;
 
       debugfile = find_separate_debug_file_by_buildid (objfile);
 
       if (debugfile == NULL)
 	debugfile = find_separate_debug_file_by_debuglink (objfile);
 
+      cleanup = make_cleanup (xfree, debugfile);
       if (debugfile)
 	{
-	  struct cleanup *cleanup = make_cleanup (xfree, debugfile);
-	  bfd *abfd = symfile_bfd_open (debugfile);
+	  abfd = symfile_bfd_open (debugfile);
+	  make_cleanup_bfd_unref (abfd);
+	}
 
+      if (abfd == NULL)
+	{
+	  abfd = find_separate_debug_file_in_section (objfile);
 	  make_cleanup_bfd_unref (abfd);
-	  symbol_file_add_separate (abfd, symfile_flags, objfile);
-	  do_cleanups (cleanup);
 	}
+
+      if (abfd != NULL)
+	symbol_file_add_separate (abfd, symfile_flags, objfile);
+
+      do_cleanups (cleanup);
     }
 
   if (symtab_create_debug)
diff --git a/gdb/testsuite/gdb.dwarf2/dw2-gnu-debugdata.c b/gdb/testsuite/gdb.dwarf2/dw2-gnu-debugdata.c
new file mode 100644
index 0000000..b8b7e8a
--- /dev/null
+++ b/gdb/testsuite/gdb.dwarf2/dw2-gnu-debugdata.c
@@ -0,0 +1,30 @@
+/* This testcase is part of GDB, the GNU debugger.
+
+   Copyright 2012 Free Software Foundation, Inc.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
+
+#include <signal.h>
+
+static int
+debugdata_function (void)
+{
+  return raise (SIGSEGV) + 1;
+}
+
+int
+main (void)
+{
+  return debugdata_function () + 1;
+}
diff --git a/gdb/testsuite/gdb.dwarf2/dw2-gnu-debugdata.exp b/gdb/testsuite/gdb.dwarf2/dw2-gnu-debugdata.exp
new file mode 100644
index 0000000..e384412
--- /dev/null
+++ b/gdb/testsuite/gdb.dwarf2/dw2-gnu-debugdata.exp
@@ -0,0 +1,98 @@
+# Copyright 2012 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+standard_testfile
+
+load_lib dwarf.exp
+if ![dwarf2_support] {
+    return 0
+}
+
+if [build_executable ${testfile}.exp $testfile] {
+    return -1
+}
+
+proc run { test cmdline } {
+    verbose "cmdline is $cmdline"
+    set result [catch "exec $cmdline" output]
+    verbose "result is $result"
+    verbose "output is $output"
+    if {$result == 0} {
+	pass $test
+	return 0
+    } else {
+	fail $test
+	return -1
+    }
+}
+
+set strip_program [transform strip]
+set nm_program [transform nm]
+
+# Extract the dynamic symbols from the main binary, there is no need
+# to also have these in the normal symbol table.
+file delete -- ${binfile}.dynsyms
+if [run "nm -D" "[transform nm] -D ${binfile} --format=posix --defined-only | awk \\{print\\ \\\$1\\} | sort > ${binfile}.dynsyms"] {
+    return -1
+}
+
+# Extract all the text (i.e. function) symbols from the debuginfo.
+file delete -- ${binfile}.funcsyms
+if [run "nm" "[transform nm] ${binfile} --format=posix --defined-only | awk \\{if(\\\$2==\"T\"||\\\$2==\"t\")print\\ \\\$1\\} | sort > ${binfile}.funcsyms"] {
+    return -1
+}
+
+# Keep all the function symbols not already in the dynamic symbol
+# table.
+file delete -- ${binfile}.keep_symbols
+if [run "comm" "comm -13 ${binfile}.dynsyms ${binfile}.funcsyms > ${binfile}.keep_symbols"] {
+    return -1
+}
+
+# Copy the full debuginfo, keeping only a minimal set of symbols and
+# removing some unnecessary sections.
+file delete -- ${binfile}.mini_debuginfo
+if [run "objcopy 1" "[transform objcopy] -S --remove-section .gdb_index --remove-section .comment --keep-symbols=${binfile}.keep_symbols ${binfile} ${binfile}.mini_debuginfo"] {
+    return -1
+}
+
+# GDB specific - we do not have split executable in advance.
+file delete -- ${binfile}.strip
+if [run "strip" "[transform strip] --strip-all -o ${binfile}.strip ${binfile}"] {
+    return -1
+}
+
+# Inject the compressed data into the .gnu_debugdata section of the
+# original binary.
+file delete -- ${binfile}.mini_debuginfo.xz
+if [run "xz" "xz ${binfile}.mini_debuginfo"] {
+    return -1
+}
+file delete -- ${binfile}.test
+if [run "objcopy 2" "[transform objcopy] --add-section .gnu_debugdata=${binfile}.mini_debuginfo.xz ${binfile}.strip ${binfile}.test"] {
+    return -1
+}
+
+clean_restart "$testfile.strip"
+
+gdb_test "p debugdata_function" \
+    {No symbol table is loaded\.  Use the "file" command\.} \
+    "no symtab"
+
+clean_restart "$testfile.test"
+
+gdb_test "p debugdata_function" \
+    { = {<text variable, no debug info>} 0x[0-9a-f]+ <debugdata_function>} \
+    "have symtab"