Repository navigation
Open
Labels
extension-modulesC modules in the Modules dirC modules in the Modules dirtopic-free-threadingtype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
Description
Bug description:
In the free-threaded build, sys._current_exceptions() iterates every thread state under stop-the-world + HEAD_LOCK and reads t->thread_id.
Lines 2808 to 2817 in ecdef17
| _PyEval_StopTheWorldAll(runtime); | |
| HEAD_LOCK(runtime); | |
| PyInterpreterState *i; | |
| for (i = runtime->interpreters.head; i != NULL; i = i->next) { | |
| _Py_FOR_EACH_TSTATE_UNLOCKED(i, t) { | |
| _PyErr_StackItem *err_info = _PyErr_GetTopmostException(t); | |
| if (err_info == NULL) { | |
| continue; | |
| } | |
| PyObject *id = PyLong_FromUnsignedLong(t->thread_id); |
But a newly started thread sets tstate->thread_id in bind_tstate() without holding HEAD_LOCK. Its PyThreadState is already on the interpreter's thread list by this point.
Lines 164 to 178 in ecdef17
| bind_tstate(PyThreadState *tstate) | |
| { | |
| assert(tstate != NULL); | |
| assert(tstate_is_alive(tstate) && !tstate->_status.bound); | |
| assert(!tstate->_status.unbound); // just in case | |
| assert(!tstate->_status.bound_gilstate); | |
| assert(tstate != gilstate_get()); | |
| assert(!tstate->_status.active); | |
| assert(tstate->thread_id == 0); | |
| assert(tstate->native_thread_id == 0); | |
| // Currently we don't necessarily store the thread state | |
| // in thread-local storage (e.g. per-interpreter). | |
| tstate->thread_id = PyThread_get_thread_ident(); |
A thread that is still attaching is not yet stop-the-world-stoppable. _PyEval_StopTheWorldAll() therefore does not pause it, and its unlocked write to tstate->thread_id races the stop-the-world reader in _PyThread_CurrentExceptions.
Reproducer:
import sys
import threading
stop = threading.Event()
def reader(): # keep stop-the-world almost always active
for _ in range(20000):
sys._current_exceptions()
def churn(): # continuously attach new threads (bind_tstate)
for _ in range(20000):
t = threading.Thread(target=lambda: None)
t.start()
t.join()
readers = [threading.Thread(target=reader) for _ in range(8)]
churners = [threading.Thread(target=churn) for _ in range(4)]
for t in readers + churners: t.start()
for t in readers + churners: t.join()TSAN Report:
WARNING: ThreadSanitizer: data race (pid=94157)
Read of size 8 at 0x0001081180b0 by thread T1:
#0 _PyThread_CurrentExceptions pystate.c:2817
#1 sys__current_exceptions sysmodule.c.h:1180
#2 cfunction_vectorcall_NOARGS methodobject.c:508
#3 PyObject_Vectorcall call.c:327
#4 _Py_VectorCallInstrumentation_StackRefSteal ceval.c:768
#5 _PyEval_EvalFrameDefault generated_cases.c.h:1906
...
Previous write of size 8 by thread T9:
#0 bind_tstate pystate.c:178
#1 _PyThreadState_Bind pystate.c:2660
#2 thread_run _threadmodule.c:384
#3 pythread_wrapper thread_pthread.h:234
SUMMARY: ThreadSanitizer: data race pystate.c:2817 in _PyThread_CurrentExceptions
CPython versions tested on:
CPython main branch
Operating systems tested on:
macOS
Linked PRs
Metadata
Metadata
Assignees
Labels
extension-modulesC modules in the Modules dirC modules in the Modules dirtopic-free-threadingtype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
Activity
[-]Data race between `sys._current_exceptions()` and a concurrently attaching thread (MacOS ARM)[/-][+]Data race between `sys._current_exceptions()` and a concurrently attaching thread[/+]zainnadeem786 commentedon Jul 1, 2026
I took a closer look at this on the latest
main(564c58c718bc) using a free-threaded Windows debug build.I couldn't independently reproduce the reported ThreadSanitizer warning because TSAN isn't available with the Windows/MSVC free-threaded build, so I focused on tracing the implementation and verifying the reported code paths.
From what I found, the synchronization concern described in the report appears consistent with the current implementation.
_PyThreadState_New()publishes the new thread state intointerp->threads.headwhile holdingHEAD_LOCK, but the child thread later calls_PyThreadState_Bind(), wherebind_tstate()initializeststate->thread_idwithoutHEAD_LOCK.Meanwhile,
_PyThread_CurrentExceptions()performs stop-the-world, acquiresHEAD_LOCK, iterates the published thread-state list, and readst->thread_id. Sincebind_tstate()runs before_PyThreadState_Attach(), my understanding is that an attaching thread is not yet fully covered by stop-the-world, so the write tothread_idmay race with those readers.While tracing this path, I also noticed that
_PyThread_CurrentFrames()andPyThreadState_SetAsyncExc()use the samethread_idpublication/read pattern, so the discussion may extend beyond_current_exceptions()alone.I also considered a few possible implementation approaches (synchronizing publication under
HEAD_LOCK, atomic publication/access, or excluding published-but-unbound thread states from these APIs), but each seems to have different lifecycle implications, so I don't want to guess at the intended design.Before attempting a PR, I'd appreciate some guidance from the runtime/free-threading maintainers on the intended invariant here.
Should published-but-not-yet-bound thread states be visible to APIs such as
_current_exceptions()and_current_frames(), or is the expectation thatthread_idpublication itself should be synchronized (for example viaHEAD_LOCKor atomic access)?If there's a preferred direction, I'd be happy to investigate further and work on a patch.
tonghuaroot commentedon Jul 1, 2026
Confirmed on a free-threaded ThreadSanitizer build (
--with-thread-sanitizer --disable-gil --with-pydebug); the reproducer fires on essentially every run:bind_tstatewritesthread_idafter thePyThreadStateis oninterp->threads.headbut before the thread attaches, so stop-the-world can't pause it and the store racessys._current_exceptions()._PyThread_CurrentFramesreads the same field but I couldn't make it race — itsif (frame == NULL) continue;gate means the thread only reaches the read afterPyEval_AcquireThreadhas synchronized throughtstate->state.Suggested labels:
topic-free-threading,interpreter-core(on top oftype-bug).Fix is small — store
thread_id/native_thread_idwith a relaxed atomic inbind_tstateand read them with a relaxed atomic load in the thread-list readers, matchingtstate->state. Opened #152755 (TSan-clean across 30+ runs).pythongh-152741: Publish thread states to the interpreter's thread li…
Merge branch 'main' into pythongh-152741-defer-tstate-publish
Merge remote-tracking branch 'upstream/main' into pythongh-152741-def…