Skip to content

AIX build failure #23962

Closed
Closed
AIX build failure#23962
Assignees
richardlau
Labels
aixIssues and PRs related to the AIX platform.

Description

@richardlau
richardlau
opened on Oct 29, 2018

Looking into this. Opening for investigation/tracking.

Refs: #22237 (comment)

https://ci.nodejs.org/job/node-test-commit-aix/18405/nodes=aix61-ppc64/consoleFull

First few errors in build log:

01:12:48 In file included from ../src/env.h:27:0,
01:12:48                  from ../src/connection_wrap.h:6,
01:12:48                  from ../src/connection_wrap.cc:1:
01:12:48 ../src/aliased_buffer.h:27:22: error: expected type-specifier
01:12:48            typename = std::enable_if_t<std::is_scalar<NativeT>::value>>
01:12:48                       ^
01:12:48 ../src/aliased_buffer.h:27:22: error: expected '>'
01:12:48 In file included from ../src/env.h:27:0,
01:12:48                  from ../src/connection_wrap.h:6,
01:12:48                  from ../src/connection_wrap.cc:1:
01:12:48 ../src/aliased_buffer.h:65:31: error: template argument 3 is invalid
01:12:48                  v8::Uint8Array>& backing_buffer)
01:12:48                                ^
01:12:48 In file included from ../src/env.h:27:0,
01:12:48                  from ../src/connection_wrap.h:6,
01:12:48                  from ../src/connection_wrap.cc:1:
01:12:48 ../src/aliased_buffer.h:123:41: error: template argument 3 is invalid
01:12:48      Reference(AliasedBuffer<NativeT, V8T>* aliased_buffer, size_t index)
01:12:48                                          ^
01:12:48 ../src/aliased_buffer.h:163:31: error: template argument 3 is invalid
01:12:48      AliasedBuffer<NativeT, V8T>* aliased_buffer_;
01:12:48

cc @nodejs/platform-aix

Activity

  1. richardlau
    added
    aixIssues and PRs related to the AIX platform.
    on Oct 29, 2018
  2. richardlau
    self-assigned this
    on Oct 29, 2018

  3. addaleax commented on Oct 29, 2018

    @addaleax
    addaleax
    on Oct 29, 2018
    Member

    Looks like this comes from #23808, on which CI was run but not on AIX?

  4. richardlau commented on Oct 29, 2018

    @richardlau
    richardlau
    on Oct 29, 2018
    MemberAuthor

    #23808 was the most recent change to aliased_buffer.h.

  5. richardlau commented on Oct 29, 2018

    @richardlau
    richardlau
    on Oct 29, 2018
    MemberAuthor

    Looks like this comes from #23808, on which CI was run but not on AIX?

    @refack @nodejs/build-infra Any ideas why the CI for #23808 (comment) (https://ci.nodejs.org/job/node-test-pull-request/18121/) didn't run on AIX?

  6. richardlau commented on Oct 29, 2018

    @richardlau
    richardlau
    on Oct 29, 2018
    MemberAuthor

    Also from the build log https://ci.nodejs.org/job/node-test-commit-aix/18405/nodes=aix61-ppc64/consoleFull it looks like gcc 4.8.5 is being used, which doesn't look right:

    00:44:45 + . ./build/jenkins/scripts/select-compiler.sh
00:44:45 + [  != DONT ]
00:44:45 + SELECT_ARCH=AIXPPC
00:44:45 + [ -z ]
00:44:45 /tmp/jenkins6046268851601114322.sh[29]: test: 0403-004 Specify a parameter with this command.
00:44:45 + [ AIXPPC = PPC64LE ]
00:44:45 + [ AIXPPC = S390X ]
00:44:45 + [ AIXPPC = AIXPPC ]
00:44:45 + echo Setting compiler for Node version  on AIX
00:44:45 Setting compiler for Node version  on AIX
00:44:45 + [  -gt 9 ]
00:44:45 + which gcc
00:44:45 + which g++
00:44:45 + which g++
00:44:45 + export CC=ccache /opt/freeware/bin/ccache/gcc CXX=ccache /opt/freeware/bin/ccache/g++ CXX_host=ccache /opt/freeware/bin/ccache/g++
00:44:45 + echo Compiler set to default at 4.8.5
00:44:45 Compiler set to default at 4.8.5

  7. richardlau commented on Oct 29, 2018

    @richardlau
    richardlau
    on Oct 29, 2018
    MemberAuthor

    I think this error suggests the script is being run with the default korn shell instead of bash?

    /tmp/jenkins6046268851601114322.sh[29]: test: 0403-004 Specify a parameter with this command.

    @nodejs/build-infra Can someone check this on the AIX CI machines?

  8. richardlau commented on Oct 29, 2018

    @richardlau
    richardlau
    on Oct 29, 2018
    MemberAuthor

    This is what the equivalent select-compiler.sh output on AIX is supposed to look like in a recent working build off master:

    https://ci.nodejs.org/job/node-test-commit-aix/18415/nodes=aix61-ppc64/consoleFull

    06:23:05 + . ./build/jenkins/scripts/select-compiler.sh
06:23:05 ++ '[' '' '!=' DONT ']'
06:23:05 ++ case $NODE_NAME in
06:23:05 ++ SELECT_ARCH=AIXPPC
06:23:05 ++ '[' -z ']'
06:23:05 +++ python tools/getnodeversion.py
06:23:05 ++ NODE_VERSION=12.0.0
06:23:05 +++ echo 12.0.0
06:23:05 +++ cut -d . -f 1
06:23:05 ++ NODEJS_MAJOR_VERSION=12
06:23:05 ++ '[' AIXPPC = PPC64LE ']'
06:23:05 ++ '[' AIXPPC = S390X ']'
06:23:05 ++ '[' AIXPPC = AIXPPC ']'
06:23:05 ++ echo 'Setting compiler for Node version 12 on AIX'
06:23:05 Setting compiler for Node version 12 on AIX
06:23:05 ++ '[' 12 -gt 9 ']'
06:23:05 ++ export LIBPATH=/home/iojs/gmake/opt/freeware/lib:/home/iojs/gcc-6.3.0-1/opt/freeware/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/pthread/ppc64:/home/iojs/gcc-6.3.0-1/opt/freeware/lib
06:23:05 ++ LIBPATH=/home/iojs/gmake/opt/freeware/lib:/home/iojs/gcc-6.3.0-1/opt/freeware/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/pthread/ppc64:/home/iojs/gcc-6.3.0-1/opt/freeware/lib
06:23:05 ++ export PATH=/home/iojs/gcc-6.3.0-1/opt/freeware/bin:/opt/freeware/bin/ccache:/home/iojs/jdk8/jre/bin:/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java6/jre/bin:/usr/java6/bin
06:23:05 ++ PATH=/home/iojs/gcc-6.3.0-1/opt/freeware/bin:/opt/freeware/bin/ccache:/home/iojs/jdk8/jre/bin:/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java6/jre/bin:/usr/java6/bin
06:23:05 +++ which gcc
06:23:05 +++ which g++
06:23:05 +++ which g++
06:23:05 ++ export 'CC=ccache /home/iojs/gcc-6.3.0-1/opt/freeware/bin/gcc' 'CXX=ccache /home/iojs/gcc-6.3.0-1/opt/freeware/bin/g++' 'CXX_host=ccache /home/iojs/gcc-6.3.0-1/opt/freeware/bin/g++'
06:23:05 ++ CC='ccache /home/iojs/gcc-6.3.0-1/opt/freeware/bin/gcc'
06:23:05 ++ CXX='ccache /home/iojs/gcc-6.3.0-1/opt/freeware/bin/g++'
06:23:05 ++ CXX_host='ccache /home/iojs/gcc-6.3.0-1/opt/freeware/bin/g++'
06:23:05 ++ echo 'Compiler set to 6.3'
06:23:05 Compiler set to 6.3

    We should be using gcc 6.3 on AIX for master (https://github.com/nodejs/node/blob/master/BUILDING.md#aix).

  9. refack commented on Oct 29, 2018

    @refack
    refack
    on Oct 29, 2018
    Contributor

    I fixed a bug in the build script around 4AM EDT at @joyeecheung's ping on IRC
    image

    tl;dr the default shell on AIX is not bash so some of the tests in select-compiler.sh weren't evaluated correctly, and the job ended up compiling with gcc4.8.

    I'm closing this as resolved, but if anyone sees one of those fails here is the explanation.

  10. refack
    closed this as completedon Oct 29, 2018

  11. refack commented on Oct 29, 2018

    @refack
    refack
    on Oct 29, 2018
    Contributor

    @refack Any ideas why the CI for #23808 (comment) (ci.nodejs.org/job/node-test-pull-request/18121) didn't run on AIX?

    There were infra issues, so the AIX job was disabled for a few days. I turned it back on last night.

  12. richardlau commented on Oct 29, 2018

    @richardlau
    richardlau
    on Oct 29, 2018
    MemberAuthor

    @refack Any ideas why the CI for #23808 (comment) (ci.nodejs.org/job/node-test-pull-request/18121) didn't run on AIX?

    There were infra issues, so the AIX job was disabled for a few days. I turned it back on last night.

    Is that logged anywhere? That means that for the few days it was disabled none of the CI's were running on AIX.

  13. refack commented on Oct 29, 2018

    @refack
    refack
    on Oct 29, 2018
    Contributor

    Is that logged anywhere? That means that for the few days it was disabled none of the CI's were running on AIX.

    I'm not sure if I sent out a notification on IRC...
    There was a notice that one machines whet offline for manual testing https://github.com/nodejs/build/projects/2

    P.S. Anyway as policy whenever we (the Build WG) take a platform offline, we take responsibility to chase down regressions afterwards, if any arise.

  14. mhdawson commented on Oct 29, 2018

    @mhdawson
    mhdawson
    on Oct 29, 2018
    Member

    @refack I don't quite understand, the last update to https://github.com/nodejs/build/commits/master/jenkins/scripts/select-compiler.sh was 27 days ago
    based on the history?

  15. refack commented on Oct 29, 2018

    @refack
    refack
    on Oct 29, 2018
    Contributor

    @mhdawson AFAICT this is what happened:

    1. select-compiler.sh has always been silently failing on AIX and picking up gcc4.8. My assumption is that it's because it uses bash features, but the the aix Jenkins job was calling the default sh which is not bash.
    2. During the last few days we landed several PRs with code that actually requires gcc4.9.
    3. The AIX job was offline during the last few days
    4. When I brought it back online (2018-10-28 ~10PM EDT) it started surfacing the issue with the wrong gcc version.
    5. At 2018-10-29 ~4AM EDT Joyee pinged the build IRC channel
    6. I made the following change; (adding an explicit bash shebang #!/usr/bin/bash -xe)
    7. Everything works (that is select-compiler.sh picks gcc6 and the code in master compiles and passed the test suite).

  16. mhdawson commented on Oct 29, 2018

    @mhdawson
    mhdawson
    on Oct 29, 2018
    Member

    Ok thanks, got it. I suspect it was just using the wrong compiler since the last update I made. I had checked the release jobs carefully, but I guess I needed to check all of them (one of the reasons I was cautious about changing the script).

    Separately, I think we should open an issue in the build repo any time we remove a platform from one of the jobs so that it gets maximum visibility.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

aixIssues and PRs related to the AIX platform.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions