Re: [CT-NG] WIP - status

[Bryan Hundven <bryanhundven at gmail dot com>]

Yann, All,

On Wed, Dec 22, 2010 at 2:59 PM, Yann E. MORIN
<yann.morin.1998@anciens.enib.fr> wrote:
> Hello All!
>
> The repository has been quiet lately, apart for some bug fixes and a few
> contributed improvements. But this does not mean developement has stalled!
>
> I'm currently working on three important things. The first impacts the
> canadian cross compilation, the second touches the companion libraries,
> while the third is about the C libraries. (In order of importance in my
> point of view, does not imply any ordering on arrival!)
>
> -------------------
> TL;DR:
> - canadian cross overhaul:
> Â- need only for existing build->target cross-toolchain
> Â- build the rest
> - companion libraries;
> Â- build our own, or rely entirely on up-to-date distros
> - C library:
> Â- simpler process, less steps
> Â- reunite glibc and eglibc
> - testing tree will be pushed shortly
>
> -------------------
> On the canadian-cross side, we currently need to have two pre-exosting
> cross-toolchains:
> Â- one to build the tools proper: runs on build, generates code for host
> Â- one to build the C library  : runs on build, generates code for target
>
> Also, in case of canadian-cross, we avoid both of the core cc passes. But
> what are the core cc good for, in the end? Building the C library for
> the target, and nothing else.
>
> So, by avoiding those two core cc, we end up having to provide a complete
> toolchain just for the prupose of building the C library. That's indeed a
> little bit deficient, as we already have code just for this!
>
> So I'm rewiring the stuff internally so that we only need the first toolchain
> (ie. build -> host), and use the core cc to build the target C library. So
> the core cc are always running on build, and generating for target, whatever
> the configuration, while the final cc is always running on host (which may
> be the same as build!) and generating for target, whatever the configuration.
>
> On the other hand, the core cc are curently used to issue the final compiler
> when we are targetting bare-metal. This means that the above is not entirely
> true, and the exception is bare-metal. But the current wrappers to pass-1 and
> pass-2 makes it easy enough to decide on whether running on build vs. on host.
>
> Still, we are missing yet a single part of the toolchain, the binary utilities
> (namely: GNU binutils, elf2flt and sstrip) that we still have to handle to
> generate the final toolchain. As we have to build the target C library, we
> need binutils to run on build and generate for target, but as part of building
> the toolchain, we also have to build them to run on host and generate for
> target.
>
> This last part means we have to build them twice in the canadian case. This is
> quite easy, as the three binary utilities are quite easy to cross-configure
> and cross-build. But it should be done with care, to avoid duplication the
> build process.
>
> Also, while we are at it, the build tools are currently wrapped into
> ${CT_PREFIX_DIR}/buildtools, and removed after the toolchain is built. This
> is defintely not their place, and they get moved into a sub-dir of
> ${CT_WORK_DIR}/${CT_TARGET} (the place where the toolchain is built). Maybe
> it could/should even go into ${CT_BUILD_DIR}. Anyway, we have to be able to
> save and restore them (as part of the build-restart feature).
>
> Although that is an almost independent issue, I decided to treat it in the
> same run, as the canadian stuff already touches PATH, and maybe also the
> place we install the core cc. So it seemed to fit.
>
> To sum up, here is what is changing on the canadian front:
> - move kernel headers just before libc headers
> - move the buildtools from ${CT_PREFIX_DIR} to ${CT_WORK_DIR}
> Âa- in a variable CT_BUILDTOOLS_DIR still pointing to ${CT_PREFIX_DIR}/buildtools
> Âb- move CT_BUILDTOOLS_DIR below ${CT_WORK_DIR}
> Âc- move comptools to ${CT_BUILDTOOLS_DIR}
> - add binutils frontend/backend
> Â- call the binutils backend with build -> host Â-> target (as done today)
> - implement the new canadian way:
> Â- add binutils frontend for   Âbuild -> build -> target
> Â- always put core_cc in PATH
>  Â- core_binutils always do   Âbuild -> build -> target
>  Â- core_cc always does     Âbuild -> build -> target
> Â Â Â- take care about cross/canadian bare-metal!
> Â- remove target tuple selection from the toolchain menu
>
> -------------------
> Next, the companion libraries are evolving. Distributions are catching up
> with the strictly required versions (they have to if they want to bundle
> the latest gccs), so this will shortly become a non-issue, and building our
> own will no longer be required.
>
> Nonetheless, there exists quite a bunch of existing stable production
> machines for which it is not sensible to upgrade the system, and we still
> have to support that. So building our own versions will still be needed.
>
> So, either the distributions are up-to-date, and provide all the required
> (or optional) companion libraries, and we use that, or the distribution does
> not provide all of them, and we build all of them. This is an all-or-nothing
> kind of choice. I do not believe there are sane cases where a distribution
> has only part of the complibs, and not the others.
>
> The complete migration process is not yet fully slated, but I have a few
> ideas already. Yet, the canadian case makes it complex, as we can't use the
> build system's complibs. So either we require them to be already built for
> running on host, or we still build our own versions.
>
> To sum up, here what is changing on the companion libraries front:
> - decorelate complibs support vs. use from gcc
> Â- configuration-wise
> Â- code wise: remove required complibs prefix, make it optional
> - add option to choose which to use
> Â- build our own, or
> Â- use those from the distribution (or am existing host 'sysroot')
>
> -------------------
> And finally, the C libraries. As of today, building a C library requires
> four steps: libc_headers, libc_start_files, libc, and libc_finish.
>
> There is no step between libc_headers and libc_start_files, so those two
> can be folded into one single step. uClibc and eglibc already does this.
> The mingw libc does nothing for start files, while newlib does nothing for
> headers. Only glibc has two separate, different code paths that hopefully
> can be reunited.
>
> The we can have only three steps: libc_start_files (which install headers
> and a few dummy/minimal .o and .so files), libc and libc_finish.
>
> libc_finish is used by glibc to post-install some utilities. This is legacy
> code dating back to long before NPTL, and the only version which still uses
> that is 2.3.6 (which is now long obsolete in crosstool-NG, but still used by
> the ia64 sample). If I can make that sample to build using a newer version,
> then we can get rid of libc_finish altogether.
>
> Which leaves us with two steps: lib_start_files and libc! :-)
>
> Finally, as eglibc is a spin-off from glibc, and regurlarly resynced from
> upstream, then we can try to reunite the two code paths that are really,
> really similar. In the process, we can play 'drop-the-legacy-code-paths'.
>
> To sum up, on the C library front:
> - fold libc_headers and libc_start_files into one single step
> - get rid of libc_finish
> - re-unite glibc and eglibc
>
> -------------------
> This is currently only on my HDD, I'll push those into a test tree later
> in the week.
>
> Thank you for your attention!
> And congratulations for those who read all down to here! ;-)

If you need any help with anything, let me know.

> Regards,
> Yann E. MORIN.
>
> --
> .-----------------.--------------------.------------------.--------------------.
> | ÂYann E. MORIN Â| Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
> | +33 662 376 056 | Software ÂDesigner | \ / CAMPAIGN Â Â | Â___ Â Â Â Â Â Â Â |
> | +33 223 225 172 `------------.-------: ÂX ÂAGAINST Â Â Â| Â\e/ ÂThere is no Â|
> | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL  Â|  v  conspiracy. Â|
> '------------------------------^-------^------------------^--------------------'
>
> --
> For unsubscribe information see http://sourceware.org/lists.html#faq
>
>


-Bryan

--
For unsubscribe information see http://sourceware.org/lists.html#faq