Re: [PATCH 1 of 3] cc/gcc: Add the ability to build gcc from svn

[Bryan Hundven <bryanhundven at gmail dot com>]

On Mon, Aug 20, 2012 at 4:16 AM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> Bryan, All,
>
> On Sunday 19 August 2012 10:14:10 Bryan Hundven wrote:
>> # HG changeset patch
>> # User Bryan Hundven <bryanhundven@gmail.com>
>> # Date 1345363489 25200
>> # Node ID 1c7363e92ede42f223edf3497fab348e16d4b05d
>> # Parent  11c23aa9c9f9e748cd6059367ef19fff23d61bd5
>> cc/gcc: Add the ability to build gcc from svn
>
> Any reason you do not use the git tree instead?
>     http://gcc.gnu.org/wiki/GitMirror
>
> There should already be some provisions for using git in
> ct-ng. See:
>     scripts/functions:687:CT_GetGit() {
>     scripts/functions:812:CT_ExtractGit()
>
> Maybe some stuff is incorrect/missing, and probably it can be enhanced...
> YMMV, as they say...

Development for gcc is on SVN.
I wouldn't expect normal users to build a toolchain from SVN, except
for users building from release branches.

In the same vain, but kind of off-topic, I think it would be nice to
have a wrapper function like:

  CT_GetSource
  CT_ExtractSource

that can take an array of urls as an argument.
For instance:

delcare -a urls

urls+=( ("git", "git://gcc.gnu.org/git/gcc.git") )
urls+=( ("svn", "svn://gcc.gnu.org/svn/gcc") )
urls+=( ("ftp",
"ftp://gcc.gnu.org/pub/gcc/releases/gcc-${VERSION}/gcc-${VERSION}.tar.bz2";)
)

CT_GetSource ${urls[${method}]}
CT_ExtractSource ${urls[${method}]}

(granted this stuff won't work, but is the general idea. Its early in
the day... need coffee...)

So that repositories held in multiple vcs formats can be obtained the
way the user prefers. (for instance, if they are a developer, they can
opt for svn and use "checkout" instead of "export" so they can test
their changes.)

>
>> I took some of the svn functionality from eglibc.
>>
>> Signed-off-by: Bryan Hundven <bryanhundven@gmail.com>
>>
>> diff -r 11c23aa9c9f9 -r 1c7363e92ede config/cc/gcc.in
>> --- a/config/cc/gcc.in        Wed Aug 15 22:06:22 2012 +0200
>> +++ b/config/cc/gcc.in        Sun Aug 19 01:04:49 2012 -0700
>> @@ -36,6 +36,12 @@
>
> Insert svn entry here, not after the comment below. That way, newer versions
> added with the scripts/addToolVersion.sh will be properly ordered (I guess
> we want SVN to always be at the top).

Good idea.

>
>>  # Don't remove next line
>>  # CT_INSERT_VERSION_BELOW
>>
>> +config CC_V_SVN
>> +    bool
>> +    prompt "gcc from svn"
>> +    depends on EXPERIMENTAL
>
> + depends on CONFIGURE_has_svn

ack.

>
>> +    select CC_GCC_latest
>> +
>>  config CC_V_linaro_4_7_2012_07
>>      bool
>>      prompt "linaro-4.7-2012.07 (EXPERIMENTAL)"
>> @@ -228,6 +234,59 @@
>>
>>  endchoice
>>
>> +if CC_V_SVN && CONFIGURE_has_svn
>
> No need for CONFIGURE_has_svn, it's hiding CC_V_SVN above.

ack.

>
>> +
>> +config GCC_BRANCH
>> +    string
>> +    prompt "Branch to use"
>
> Branch or tag to use (see below)

ack.

>
>> +    default "trunk"
>> +    help
>> +      Enter the branch of gcc to use.
>> +      Default is trunk
>
> Add a few examples:
>   eg.: branches/gcc-4_7-branch, branches/gcc-in-cxx, branches/x32...
>
> And I'd state that the full path to the branch should be specified. That
> way, it is possible to use any reference (branch, level-2 branch or tag).

Gotcha. That would make the fetch code in gcc.sh a little cleaner.

>
>> +      You can find other branches here:
>> +       http://gcc.gnu.org/viewcvs/branches/
>
> Probably, the persons that will be using this feature know what branch
> they'll want to use. Using svn/git is not for the faint of heart! ;-)
> So, I'd remove that last part of the comment.

ack.

>
>> +config GCC_REVISION
>> +    string
>> +    prompt "Revision to use"
>> +    default "HEAD"
>> +    help
>> +      Enter the revision of trunk you want to use.
>> +      Default is HEAD
>> +
>> +      A revision argument can be one of:
>> +          NUMBER       revision number
>> +          '{' DATE '}' revision at start of the date (*)
>> +          'HEAD'       latest in repository
>> +
>> +      (*) If you want to use a date, please use ISO-8601 formats if
>> +      at all possible.
>> +
>> +config GCC_HTTP
>> +    bool
>> +    prompt "use http:// instead of svn://"
>> +    help
>> +      By default, when gcc is downloaded it is checked out using
>> +      svn://gcc.gnu.org/svn/gcc. This option allows you to download gcc
>> +      from http://gcc.gnu.org/svn/gcc, if you are behind a proxy or firewall.
>> +      If you are behind a proxy, don't forget to update your
>> +      .subversion/servers file with your proxy info in [global].
>> +
>> +config GCC_CHECKOUT
>> +    bool
>> +    prompt "checkout instead of export"
>> +    default n
>> +    help
>> +      By default, the gcc download will be an export of the subversion
>> +      repository. If you say 'y' here, then the repository will instead be
>> +      checked-out, so that you can update it later.
>> +
>> +      Note that crosstool-NG will *not* update your working copy, you will
>> +      have to do that yourself.
>> +
>> +endif
>> +
>>  config CC_GCC_4_2
>>      bool
>>      select CC_GCC_4_2_or_later
>> @@ -301,6 +360,10 @@
>>      bool
>>      select CC_GCC_4_6_or_later
>>
>> +config CC_GCC_latest
>> +    bool
>> +    select CC_GCC_4_7
>
> - select CC_GCC_4_7
> + select CC_GCC_4_7_or_later
> + select CC_GCC_HAS_GRAPHITE
> + select CC_GCC_HAS_LTO
> ... and so on...

Ok, thanks. I wasn't sure how you'd want that.

>> +
>>  config CC_GCC_HAS_GRAPHITE
>>      bool
>>
>> @@ -364,6 +427,7 @@
>>      string
>>  # Don't remove next line
>>  # CT_INSERT_VERSION_STRING_BELOW
>> +    default "SVN" if CC_V_SVN
>>      default "linaro-4.7-2012.07" if CC_V_linaro_4_7_2012_07
>>      default "4.7.1" if CC_V_4_7_1
>>      default "4.7.0" if CC_V_4_7_0
>> diff -r 11c23aa9c9f9 -r 1c7363e92ede scripts/build/cc/gcc.sh
>> --- a/scripts/build/cc/gcc.sh Wed Aug 15 22:06:22 2012 +0200
>> +++ b/scripts/build/cc/gcc.sh Sun Aug 19 01:04:49 2012 -0700
>> @@ -4,31 +4,53 @@
>>
>>  # Download gcc
>>  do_cc_get() {
>> -    local linaro_version
>> -    local linaro_series
>> -    local linaro_base_url="http://launchpad.net/gcc-linaro";
>> +    if [ -n ${CC_V_SVN} ]; then
>> +        # Get gcc from SVN!
>> +        local svn_base
>> +        if [ "${CT_GCC_HTTP}" = "y" ]; then
>> +            svn_base="http://gcc.gnu.org/svn/gcc";
>> +        else
>> +            svn_base="svn://gcc.gnu.org/svn/gcc"
>> +        fi
>> +
>> +        case "${CT_GCC_BRANCH}" in
>> +            trunk)  svn_base+="/trunk";;
>> +            *)      svn_base+="/branches/${CT_GCC_BRANCH}";;
>> +        esac
>
> case-esac undeed, the full patch nust be specified in the configuration.
> (see above)

Yup. I commented on that above.

>
>> +
>> +        CT_CC_VERSION="${CT_GCC_BRANCH}"
>
> Of course, '/' in CT_GCC_BRANCH should be 's:/:_:g'

Oh, hehe. Good catch!

>
>> +        CT_GetSVN "gcc-${CT_GCC_BRANCH}" \
>> +            "${svn_base}/" \
>> +            "${CT_GCC_REVISION:-HEAD}"
>> +    else
>> +        # Get gcc from a released tarball!
>> +        local linaro_version
>> +        local linaro_series
>> +        local linaro_base_url="http://launchpad.net/gcc-linaro";
>>
>> -    # Account for the Linaro versioning
>> -    linaro_version="$( echo "${CT_CC_VERSION}"      \
>> -                       |sed -r -e 's/^linaro-//;'   \
>> -                     )"
>> -    linaro_series="$( echo "${linaro_version}"      \
>> -                      |sed -r -e 's/-.*//;'         \
>> -                    )"
>>
>> -    # Ah! gcc folks are kind of 'different': they store the tarballs in
>> -    # subdirectories of the same name! That's because gcc is such /crap/ that
>> -    # it is such /big/ that it needs being splitted for distribution! Sad. :-(
>> -    # Arrgghh! Some of those versions does not follow this convention:
>> -    # gcc-3.3.3 lives in releases/gcc-3.3.3, while gcc-2.95.* isn't in a
>> -    # subdirectory! You bastard!
>> -    CT_GetFile "gcc-${CT_CC_VERSION}"                                                       \
>> -               {ftp,http}://ftp.gnu.org/gnu/gcc{,{,/releases}/gcc-${CT_CC_VERSION}}         \
>> -               ftp://ftp.irisa.fr/pub/mirrors/gcc.gnu.org/gcc/releases/gcc-${CT_CC_VERSION} \
>> -               ftp://ftp.uvsq.fr/pub/gcc/snapshots/${CT_CC_VERSION}                         \
>> -               "${linaro_base_url}/${linaro_series}/${linaro_version}/+download"
>> +        # Account for the Linaro versioning
>> +        linaro_version="$( echo "${CT_CC_VERSION}"      \
>> +                           |sed -r -e 's/^linaro-//;'   \
>> +                         )"
>> +        linaro_series="$( echo "${linaro_version}"      \
>> +                          |sed -r -e 's/-.*//;'         \
>> +                        )"
>>
>> +        # Ah! gcc folks are kind of 'different': they store the tarballs in
>> +        # subdirectories of the same name! That's because gcc is such /crap/ that
>> +        # it is such /big/ that it needs being splitted for distribution! Sad. :-(
>> +        # Arrgghh! Some of those versions does not follow this convention:
>> +        # gcc-3.3.3 lives in releases/gcc-3.3.3, while gcc-2.95.* isn't in a
>> +        # subdirectory! You bastard!
>
> Well, I guess all the insanities *I* did put in there in the early days
> (ie. when nothing would work as expected, and I would get heated quite
> fast) should now go... Tatata... Shame on me...
>
> I'll address this issue myself.

I just changed the indentation. I kept all the "insanities" intact. :D

>
>> +        CT_GetFile "gcc-${CT_CC_VERSION}"                                                       \
>> +                   {ftp,http}://ftp.gnu.org/gnu/gcc{,{,/releases}/gcc-${CT_CC_VERSION}}         \
>> +                   ftp://ftp.irisa.fr/pub/mirrors/gcc.gnu.org/gcc/releases/gcc-${CT_CC_VERSION} \
>> +                   ftp://ftp.uvsq.fr/pub/gcc/snapshots/${CT_CC_VERSION}                         \
>> +                   "${linaro_base_url}/${linaro_series}/${linaro_version}/+download"
>> +
>> +    fi # -n CC_V_SVN
>>      # Starting with GCC 4.3, ecj is used for Java, and will only be
>>      # built if the configure script finds ecj.jar at the top of the
>>      # GCC source tree, which will not be there unless we get it and
>> @@ -41,14 +63,16 @@
>>
>>  # Extract gcc
>>  do_cc_extract() {
>> -    CT_Extract "gcc-${CT_CC_VERSION}"
>> -    CT_Patch "gcc" "${CT_CC_VERSION}"
>> +    if [ -z "${CC_V_SVN}" ]; then
>> +        CT_Extract "gcc-${CT_CC_VERSION}"
>> +        CT_Patch "gcc" "${CT_CC_VERSION}"
>>
>> -    # Copy ecj-latest.jar to ecj.jar at the top of the GCC source tree
>> -    if [ "${CT_CC_LANG_JAVA_USE_ECJ}" = "y"                     \
>> -         -a ! -f "${CT_SRC_DIR}/gcc-${CT_CC_VERSION}/ecj.jar"   \
>> -       ]; then
>> -        CT_DoExecLog ALL cp -v "${CT_TARBALLS_DIR}/ecj-latest.jar" "${CT_SRC_DIR}/gcc-${CT_CC_VERSION}/ecj.jar"
>> +        # Copy ecj-latest.jar to ecj.jar at the top of the GCC source tree
>> +        if [ "${CT_CC_LANG_JAVA_USE_ECJ}" = "y"                     \
>> +             -a ! -f "${CT_SRC_DIR}/gcc-${CT_CC_VERSION}/ecj.jar"   \
>> +           ]; then
>> +            CT_DoExecLog ALL cp -v "${CT_TARBALLS_DIR}/ecj-latest.jar" "${CT_SRC_DIR}/gcc-${CT_CC_VERSION}/ecj.jar"
>> +        fi
>>      fi
>>  }
>
> Otherwise, untested, but looks good.

I tested it with trunk. I will fix this up and test branches, tags and
revisions.

I also just noticed that I have added the config option for "checkout"
or "export" but do not handle it. That will be in v2 as well.

Thank you for the review, Yann!

-Bryan

>
> Regards,
> Yann E. MORIN.
>
> --
> .-----------------.--------------------.------------------.--------------------.
> |  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
> | +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
> | +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
> | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
> '------------------------------^-------^------------------^--------------------'

--
For unsubscribe information see http://sourceware.org/lists.html#faq