Repository navigation
Description
Bug report
Bug description:
whichmodule (Modules/_pickle.c) iterates over sys.modules using PyDict_Next and immediately Py_INCREFs the borrowed references it returns, with no synchronisation. In a free-threaded build another thread can remove an entry from sys.modules between the PyDict_Next call and the subsequent Py_INCREFs. PySys_GetAttr gives a strong reference to the dict itself, so the dict won't be freed, but the borrowed key/value pointers returned by PyDict_Next are not protected. The path is triggered when pickling a class whose __module__ is None (e.g. numpy ufuncs).
import pickle, sys, threading, types
class NoModule:
pass
NoModule.__module__ = None # forces whichmodule to scan sys.modules
def _pickle_worker():
for _ in range(200_000):
try: pickle.dumps(NoModule)
except Exception: pass
def _mutate_worker():
for i in range(200_000):
name = f"__tmp_{i % 200}"
sys.modules[name] = types.ModuleType(name)
sys.modules.pop(f"__tmp_{(i - 50) % 200}", None)
threads = [threading.Thread(target=_pickle_worker) for _ in range(4)]
threads += [threading.Thread(target=_mutate_worker)]
for t in threads: t.start()
for t in threads: t.join()gives
$ ./python.exe ../repro.py
zsh: segmentation fault ./python.exe ./repro.pyProposed fix
Wrap the PyDict_Next loop in Py_BEGIN_CRITICAL_SECTION(modules) / Py_END_CRITICAL_SECTION, restructuring early returns as breaks so the section is always exited cleanly (same pattern used in #145446).
CPython versions tested on:
CPython main branch
Operating systems tested on:
macOS
Linked PRs
- gh-150157: fix: critical section for
PyDict_Nextin_pickle.c#150158 - [3.14] gh-150157: Fix critical section for PyDict_Next() in _pickle.c (GH-150158) #150710
- [3.15] gh-150157: Fix critical section for PyDict_Next() in _pickle.c (GH-150158) #150711
- [3.13] gh-150157: add critical section for
PyDict_Nextin_pickle.c#150951
Metadata
Metadata
Assignees
Labels
Projects
- StatusShow more project fieldsDone
Activity
gh-150157: Fix critical section for PyDict_Next() in _pickle.c (GH-15…
[3.14] gh-150157: Fix critical section for PyDict_Next() in _pickle.c (…
[3.15] gh-150157: Fix critical section for PyDict_Next() in _pickle.c (…