You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
version 6.2.1 always got Segment Fault error and crash #7454
Version:6.2.1, compile from source code and install use make
Platform:Ubuntu 12.04 and 14.04
Subsystem:
With node upgraded to 6.2.1, we got 'Segment Fault error' in several projects. It doesn't happen every time, but the probability is rather high. We've tried 'npm rebuild' according to google search result implying this may have something to do with node native modules. The error still remains.
What's your advice?
Do your projects use native add-ons (find node_modules/ -name \*.node)? If so, start by excluding those.
If crashes still happen, try collecting a backtrace. Turn on core dumps (ulimit -c unlimited) and after a crash, inspect it in gdb: gdb node core, then type thread apply all backtrace full.
Core dumps are normally written to the current working directory but your distro can reconfigure that, see sysctl kernel.core_pattern.
@bnoordhuis I have done that, below is all track, what's the matter?
Thread 10 (Thread 0x7fb7edc25700 (LWP 3370)):
#0 0x00007fb7eeff4fd0 in sem_wait ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x0000000001131ea8 in v8::base::Semaphore::Wait() ()
No symbol table info available.
#2 0x0000000000fce039 in v8::platform::TaskQueue::GetNext() ()
No symbol table info available.
#3 0x0000000000fce18c in v8::platform::WorkerThread::Run() ()
No symbol table info available.
#4 0x0000000001133190 in v8::base::ThreadEntry(void*) ()
No symbol table info available.
#5 0x00007fb7eefeee9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#6 0x00007fb7eed1b8bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#7 0x0000000000000000 in ?? ()
No symbol table info available.
---Type <return> to continue, or q <return> to quit---
Thread 9 (Thread 0x7fb7deffd700 (LWP 3375)):
#0 0x00007fb7eeff2d84 in pthread_cond_wait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x0000000001128509 in uv_cond_wait (cond=<optimised out>,
mutex=<optimised out>) at ../deps/uv/src/unix/thread.c:443
No locals.
#2 0x0000000001119868 in worker (arg=<optimised out>)
at ../deps/uv/src/threadpool.c:75
w = <optimised out>
q = <optimised out>
#3 0x0000000001128031 in uv__thread_start (arg=<optimised out>)
at ../deps/uv/src/unix/thread.c:52
ctx_p = <optimised out>
ctx = {entry = 0x1119820 <worker>, arg = 0x0}
#4 0x00007fb7eefeee9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#5 0x00007fb7eed1b8bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
---Type <return> to continue, or q <return> to quit---
No symbol table info available.
#6 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 8 (Thread 0x7fb7df7fe700 (LWP 3374)):
#0 0x00007fb7eeff2d84 in pthread_cond_wait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x0000000001128509 in uv_cond_wait (cond=<optimised out>,
mutex=<optimised out>) at ../deps/uv/src/unix/thread.c:443
No locals.
#2 0x0000000001119868 in worker (arg=<optimised out>)
at ../deps/uv/src/threadpool.c:75
w = <optimised out>
q = <optimised out>
#3 0x0000000001128031 in uv__thread_start (arg=<optimised out>)
at ../deps/uv/src/unix/thread.c:52
ctx_p = <optimised out>
ctx = {entry = 0x1119820 <worker>, arg = 0x0}
#4 0x00007fb7eefeee9a in start_thread ()
---Type <return> to continue, or q <return> to quit---
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#5 0x00007fb7eed1b8bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#6 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 7 (Thread 0x7fb7dffff700 (LWP 3373)):
#0 0x00007fb7eeff2d84 in pthread_cond_wait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x0000000001128509 in uv_cond_wait (cond=<optimised out>,
mutex=<optimised out>) at ../deps/uv/src/unix/thread.c:443
No locals.
#2 0x0000000001119868 in worker (arg=<optimised out>)
at ../deps/uv/src/threadpool.c:75
w = <optimised out>
q = <optimised out>
#3 0x0000000001128031 in uv__thread_start (arg=<optimised out>)
at ../deps/uv/src/unix/thread.c:52
---Type <return> to continue, or q <return> to quit---
ctx_p = <optimised out>
ctx = {entry = 0x1119820 <worker>, arg = 0x0}
#4 0x00007fb7eefeee9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#5 0x00007fb7eed1b8bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#6 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 6 (Thread 0x7fb7ec922700 (LWP 3372)):
#0 0x00007fb7eeff2d84 in pthread_cond_wait@@GLIBC_2.3.2 ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x0000000001128509 in uv_cond_wait (cond=<optimised out>,
mutex=<optimised out>) at ../deps/uv/src/unix/thread.c:443
No locals.
#2 0x0000000001119868 in worker (arg=<optimised out>)
at ../deps/uv/src/threadpool.c:75
w = <optimised out>
---Type <return> to continue, or q <return> to quit---
q = <optimised out>
#3 0x0000000001128031 in uv__thread_start (arg=<optimised out>)
at ../deps/uv/src/unix/thread.c:52
ctx_p = <optimised out>
ctx = {entry = 0x1119820 <worker>, arg = 0x0}
#4 0x00007fb7eefeee9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#5 0x00007fb7eed1b8bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#6 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 5 (Thread 0x7fb7ed424700 (LWP 3371)):
#0 0x00007fb7eeff4fd0 in sem_wait ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x0000000001131ea8 in v8::base::Semaphore::Wait() ()
No symbol table info available.
#2 0x0000000000fce039 in v8::platform::TaskQueue::GetNext() ()
---Type <return> to continue, or q <return> to quit---
No symbol table info available.
#3 0x0000000000fce18c in v8::platform::WorkerThread::Run() ()
No symbol table info available.
#4 0x0000000001133190 in v8::base::ThreadEntry(void*) ()
No symbol table info available.
#5 0x00007fb7eefeee9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#6 0x00007fb7eed1b8bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#7 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 4 (Thread 0x7fb7ee426700 (LWP 3369)):
#0 0x00007fb7eeff4fd0 in sem_wait ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x0000000001131ea8 in v8::base::Semaphore::Wait() ()
No symbol table info available.
#2 0x0000000000fce039 in v8::platform::TaskQueue::GetNext() ()
---Type <return> to continue, or q <return> to quit---
No symbol table info available.
#3 0x0000000000fce18c in v8::platform::WorkerThread::Run() ()
No symbol table info available.
#4 0x0000000001133190 in v8::base::ThreadEntry(void*) ()
No symbol table info available.
#5 0x00007fb7eefeee9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#6 0x00007fb7eed1b8bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#7 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 3 (Thread 0x7fb7eec27700 (LWP 3368)):
#0 0x00007fb7eeff4fd0 in sem_wait ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x0000000001131ea8 in v8::base::Semaphore::Wait() ()
No symbol table info available.
#2 0x0000000000fce039 in v8::platform::TaskQueue::GetNext() ()
---Type <return> to continue, or q <return> to quit---
No symbol table info available.
#3 0x0000000000fce18c in v8::platform::WorkerThread::Run() ()
No symbol table info available.
#4 0x0000000001133190 in v8::base::ThreadEntry(void*) ()
No symbol table info available.
#5 0x00007fb7eefeee9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#6 0x00007fb7eed1b8bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#7 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 2 (Thread 0x7fb7f0054700 (LWP 3367)):
#0 0x00007fb7eeff4fd0 in sem_wait ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1 0x0000000001128368 in uv_sem_wait (sem=0x1b85da0)
at ../deps/uv/src/unix/thread.c:330
r = <optimised out>
---Type <return> to continue, or q <return> to quit---
#2 0x0000000000f5ffe2 in node::DebugSignalThreadMain(void*) ()
No symbol table info available.
#3 0x00007fb7eefeee9a in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#4 0x00007fb7eed1b8bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 1 (Thread 0x7fb7f0046740 (LWP 3366)):
#0 0x0000000000dd2c09 in v8::internal::Runtime::CreateArrayLiteralBoilerplate(v8::internal::Isolate*, v8::internal::Handle<v8::internal::LiteralsArray>, v8::internal::Handle<v8::internal::FixedArray>, bool) ()
No symbol table info available.
#1 0x0000000000dd3deb in v8::internal::CreateArrayLiteralImpl(v8::internal::Isolate*, v8::internal::Handle<v8::internal::LiteralsArray>, int, v8::internal::Handle<v8::internal::FixedArray>, int) ()
No symbol table info available.
#2 0x0000000000dd4c70 in v8::internal::Runtime_CreateArrayLiteral(int, v8::inte---Type <return> to continue, or q <return> to quit---
rnal::Object**, v8::internal::Isolate*) ()
No symbol table info available.
#3 0x000026658380961b in ?? ()
No symbol table info available.
#4 0x00007fffecba0b50 in ?? ()
No symbol table info available.
#5 0x0000266583809561 in ?? ()
No symbol table info available.
#6 0x00007fffecba0af0 in ?? ()
No symbol table info available.
#7 0x00007fffecba0b68 in ?? ()
No symbol table info available.
#8 0x0000266583ba590b in ?? ()
No symbol table info available.
#9 0x0000000300000000 in ?? ()
No symbol table info available.
#10 0x00003d4783904139 in ?? ()
No symbol table info available.
#11 0x0000000100000000 in ?? ()
No symbol table info available.
---Type <return> to continue, or q <return> to quit---
#12 0x000037441c023749 in ?? ()
No symbol table info available.
#13 0x00007fffecba0ca8 in ?? ()
No symbol table info available.
#14 0x00003d47839ac529 in ?? ()
No symbol table info available.
#15 0x000037441c023719 in ?? ()
No symbol table info available.
#16 0x000037441c01d679 in ?? ()
No symbol table info available.
#17 0x000037441c023749 in ?? ()
No symbol table info available.
#18 0x000037441c01d679 in ?? ()
No symbol table info available.
#19 0x00007fffecba0ba0 in ?? ()
No symbol table info available.
#20 0x000026658380d157 in ?? ()
No symbol table info available.
#21 0x000037441c01d731 in ?? ()
No symbol table info available.
---Type <return> to continue, or q <return> to quit---
#22 0x00003d47839e3fc1 in ?? ()
No symbol table info available.
#23 0x0000000300000000 in ?? ()
No symbol table info available.
#24 0x000037441c023749 in ?? ()
No symbol table info available.
#25 0x0000000b00000000 in ?? ()
No symbol table info available.
#26 0x00007fffecba0c00 in ?? ()
No symbol table info available.
#27 0x0000266583b9db1f in ?? ()
No symbol table info available.
#28 0x000037441c022361 in ?? ()
No symbol table info available.
#29 0x0000000000000000 in ?? ()
No symbol table info available.
It's probably a bad pointer dereference but I can't quite place it.
%r13 corresponds to the fourth argument (the function prologue moves %rcx to it), which is a bool. 0x0(%r13) doesn't make sense in that context because it's basically a nullptr dereference. That it doesn't crash until a few instructions later probably means I read it wrong but I don't see how.
Did you have any luck with the debug binary? Do you get crashes with the v6.2.1 release binary from nodejs.org?
@bnoordhuis Thanks a lot, so you mean I should compile a binary for debugging, current node version I used is 6.2.1 and it is compiled from source. I'll try to compile a binary for debugging, but do you have any idea to quickly resolve the problem because many of our projects in production environment crashed and we really worry about this bug.
I don't have a clear picture as to the cause yet so I'm afraid I can't offer any real advice at this point. You could try downgrading to the latest v4 release and see if that resolves or ameliorates it.
@bnoordhuis Actually, previously I tried 5.X version but it has a critical problem in garbage collecting. Maybe I need try 4.X.
But, this is also a very critical problem, didn't anybody run into it? Can I at somebody else who are the core developers in nodejs team?
Activity
bnoordhuis commentedon Jun 28, 2016
Do your projects use native add-ons (
find node_modules/ -name \*.node)? If so, start by excluding those.If crashes still happen, try collecting a backtrace. Turn on core dumps (
ulimit -c unlimited) and after a crash, inspect it in gdb:gdb node core, then typethread apply all backtrace full.Core dumps are normally written to the current working directory but your distro can reconfigure that, see
sysctl kernel.core_pattern.mike442144 commentedon Jun 28, 2016
@bnoordhuis Thanks a lot, firstly, I have no native add-ons in my project, and then we've got a core dump, let me try
thread apply all backtrace full.mike442144 commentedon Jun 28, 2016
@bnoordhuis I have done that, below is all track, what's the matter?
bnoordhuis commentedon Jun 28, 2016
Thanks. Is it possible for you to try a debug build? You can compile one with
make -j8 -C out BUILDTYPE=Debug. The binary is placed inout/Debug.As a quick sanity check, can you open the core file with gdb and run this?
mike442144 commentedon Jun 28, 2016
bnoordhuis commentedon Jun 28, 2016
Thanks. The offending code appears to be this:
It's probably a bad pointer dereference but I can't quite place it.
%r13 corresponds to the fourth argument (the function prologue moves %rcx to it), which is a bool. 0x0(%r13) doesn't make sense in that context because it's basically a nullptr dereference. That it doesn't crash until a few instructions later probably means I read it wrong but I don't see how.
Did you have any luck with the debug binary? Do you get crashes with the v6.2.1 release binary from nodejs.org?
mike442144 commentedon Jun 28, 2016
@bnoordhuis Thanks a lot, so you mean I should compile a binary for debugging, current node version I used is 6.2.1 and it is compiled from source. I'll try to compile a binary for debugging, but do you have any idea to quickly resolve the problem because many of our projects in production environment crashed and we really worry about this bug.
bnoordhuis commentedon Jun 28, 2016
I don't have a clear picture as to the cause yet so I'm afraid I can't offer any real advice at this point. You could try downgrading to the latest v4 release and see if that resolves or ameliorates it.
mike442144 commentedon Jun 28, 2016
@bnoordhuis Actually, previously I tried 5.X version but it has a critical problem in garbage collecting. Maybe I need try 4.X.
But, this is also a very critical problem, didn't anybody run into it? Can I
atsomebody else who are the core developers in nodejs team?mike442144 commentedon Jun 28, 2016
@bnoordhuis I've done with compiling debug version, what's next? use it to run my project?
bnoordhuis commentedon Jun 28, 2016
Yes. I would like to know if the backtrace from the core dump is the same and whether the
--verify_heapflag makes a difference.Hey, what do you think I am?
mike442144 commentedon Jun 29, 2016
@bnoordhuis haha, let me try first.
mike442144 commentedon Jun 29, 2016
@bnoordhuis I got below message using debug version:
bnoordhuis commentedon Jun 29, 2016
Thanks, that is useful info. I believe I have an inkling of the cause.
Does
--nocrankshaftmake a difference? Does the backtrace in gdb look the same? Let me know if there is a way for me to easily reproduce the crash.24 remaining items
mike442144 commentedon Jul 18, 2016
@bnoordhuis Hi, Ben, Do I have to wait new 6.x version release?
bnoordhuis commentedon Jul 18, 2016
Yes, see linked pull request #7633.
mike442144 commentedon Jul 18, 2016
@bnoordhuis Okay, I see. thanks for your effort.
deps: back-port d721121 from v8 upstream
deps: back-port d721121 from v8 upstream
deps: back-port d721121 from v8 upstream
deps: back-port d721121 from v8 upstream