Re: New feature "source-id"

[Mark Wielaard <mjw at redhat dot com>]

On Tue, 2014-03-18 at 15:00 +0100, Gerhard Gappmeier wrote:
> On Tuesday, March 18, 2014 02:22:04 PM you wrote:
> > On Sat, 2014-03-15 at 11:49 +0100, Gerhard Gappmeier wrote:
> > That way you can use the build-id from the ELF note section to retrieve
> > both the separate .debug files and the corresponding source files. And
> > on my distro gdb even helpfully suggests how to do this:
> > Missing separate debuginfos, use: debuginfo-install at-3.1.13-14.fc20.x86_64
> > Which will then fetch the debuginfo package and all dependencies so gdb can
> > find the .debug files and the corresponding source code those .debug files
> > refer to. I don't know if the debuginfo-install suggestion is upstream or
> > only in the distro package of gdb.
> 
> If I understood this right, this means whenever a software is built the 
> sources get archived with the debug symbols in an debuginfo RPM file.
> This way the build-id is all you need to get the correct sources and debug 
> symbols.

Indeed. Just turn the build-id into the package, either through
something like https://darkserver.fedoraproject.org/ or through yum
install /usr/lib/debug/.build-id/b7/07011ecdbd5bcb1fad73cdc9b4433c791d8328.debug or just through debuginfo-install and you get both the .debug files and all sources files that .debug file refers to.

> However my idea is somewhat different and a little bit smarter IMO:
> * The SHA1 id of a git repo gets stored in the source-id meta info when 
> building.
> * There is no need of archiving the source files in RPM, deb, tar.gz or zip 
> files. We have them already in the version control system and we don't want to 
> duplicate the data
> * This solution is independent from any package format.
> * You can analyze coredumps of executables that you don't have on your system. 
> There is no need to install any RPM package for that. This way you can analyze 
> e.g. a crash within a Ubuntu package on a Fedora system.
> * The fetch-script fetches only the sources required by GDB, not the complete 
> project.

Some of those features are already possible with the way distros package
the debuginfo files. But your way might indeed be more flexible. I am
mostly wondering how to take advantage of the way distros do it
currently in your scheme. How do you describe the default distro setup
and how do you make sure not to duplicate the storage of source files?

One difference with your scheme is that the distros packages the
post-processed source files. That means they are the actual files,
however generated, that the compiler compiled to object code. Not
necessarily the pristine source files. That is so in a debugger you can
step through the source file as seen by the compiler (e.g. it will
include source files generated by configure or the lex and yacc
generated files that the compiler builds).

> > > * We need to make the new section ".note.gnu.source-id" official. I don't
> > > know who maintains this and this needs to be registered somewhere.
> > > [...]
> > > * adding file hashes (SHA1) for each source file to the debug info. This
> > > way we can completely remove the mtime check and replace it with a check
> > > of the SHA1 sum. When we can replace the existing warning with a message
> > > like "The source file does not match the executable."
> > 
> > For DWARF5 there is a proposal to add the MD5 digest to debug-line file
> > table: http://dwarfstd.org/ShowIssue.php?issue=130701.1
> > 
> > Would that be a good alternative location to store the hash of the
> > source file?
> That's exactly what I proposed. Only that I proposed SHA1 instead of MD5, but 
> this doesn't matter.
> If this is already in the DWARD standard we should use this feature and don't 
> reinvent the wheel.

It is currently just a proposal for DWARF5. The proposal deadline is end
of this month. I just reviewed that proposal and saw that it is not very
extensible, so I suggested some additions. See the discussion here:
http://thread.gmane.org/gmane.comp.standards.dwarf/100

Cheers,

Mark