Skip to content

Fix crashes during interpreter shutdown on all Python versions (3.2.6)#500

Open
nbouvrette wants to merge 1 commit intopython-greenlet:maint/3.2from
nbouvrette:fix/safe-getcurrent-during-finalization-3.2
Open

Fix crashes during interpreter shutdown on all Python versions (3.2.6)#500
nbouvrette wants to merge 1 commit intopython-greenlet:maint/3.2from
nbouvrette:fix/safe-getcurrent-during-finalization-3.2

Conversation

@nbouvrette
Copy link
Contributor

@nbouvrette nbouvrette commented Mar 12, 2026

Summary

Backport of PR #499 to maint/3.2 for greenlet 3.2.6, targeting Python 3.9-3.13.

The previous backport (3.2.5 / PR #495) only guarded Python < 3.11 (#if !GREENLET_PY311), but the vulnerability exists on all Python versions: Py_IsFinalizing() is set AFTER atexit handlers complete inside Py_FinalizeEx. This PR removes all version guards and makes the fix unconditional.

Design

Two independent guards now protect all shutdown phases:

  1. g_greenlet_shutting_down — an atexit handler registered at module init (LIFO = runs first) sets this flag. Covers the atexit phase of Py_FinalizeEx, where Py_IsFinalizing() is still False on all Python versions.

  2. Py_IsFinalizing() — covers the GC collection and later phases of Py_FinalizeEx. A compatibility shim maps to _Py_IsFinalizing() on Python < 3.13.

These guards are checked in mod_getcurrent, PyGreenlet_GetCurrent, GreenletChecker, MainGreenletExactChecker, ContextExactChecker, clear_deleteme_list(), ThreadState::~ThreadState(), _green_dealloc_kill_started_non_main_greenlet, and ThreadState_DestroyNoGIL::AddPendingCall.

What changed vs the previous PR #500

The previous version of this PR had all guards wrapped in #if !GREENLET_PY311, making them completely inactive on Python 3.11-3.13. This update:

  1. Removes all #if !GREENLET_PY311 guards — fixes are now unconditional across Python 3.9-3.13
  2. Adds type checker guardsGreenletChecker, MainGreenletExactChecker, ContextExactChecker now have shutdown protection (these were missing entirely)
  3. Adds 5 TDD-certified regression tests — verified RED on greenlet 3.3.2 (UNGUARDED) and GREEN with the fix across Python 3.10-3.14
  4. Strengthens 3 smoke tests — assert getcurrent() still returns valid objects before greenlet's cleanup (guards against over-blocking)

What changed (files)

C++ shutdown guards (8 files)

File Change
PyModule.cpp g_greenlet_shutting_down + atexit handler made unconditional
CObjects.cpp PyGreenlet_GetCurrent guard made unconditional
PyGreenlet.cpp murder_in_place() guard made unconditional, added g_greenlet_shutting_down
TThreadState.hpp clear_deleteme_list() + destructor guards made unconditional
TThreadStateDestroy.cpp AddPendingCall guard extended with g_greenlet_shutting_down
greenlet.cpp Atexit handler registration made unconditional
greenlet_refs.hpp NEW: Guards in GreenletChecker + ContextExactChecker
greenlet_internal.hpp NEW: Guard in MainGreenletExactChecker

Additional hardening

  • clear_deleteme_list() uses std::swap (zero-allocation) instead of copying the PythonAllocator-backed vector
  • The deleteme vector uses std::allocator (system malloc) instead of PyMem_Malloc
  • ThreadState uses std::malloc/std::free instead of PyObject_Malloc
  • clear_deleteme_list() preserves any pending Python exception around its cleanup loop

Tests (3 files)

  • 5 new TDD-certified regression tests in test_interpreter_shutdown.py — verified RED on greenlet 3.3.2 and GREEN with fix across Python 3.10-3.14
  • 3 strengthened smoke tests — assert getcurrent() still returns valid objects before cleanup
  • Updated file docstring and section headers — organized 21 tests into 4 documented groups
  • Fixed test_dealloc_catches_GreenletExit_throws_other — use sys.unraisablehook instead of stderr capture (pytest compatibility)
  • Fixed test_version — skip gracefully on old setuptools

Other

  • SPDX license identifier fixed: Python-2.0PSF-2.0
  • Flaky USS memory test tolerance on Windows

TDD verification

Python greenlet 3.3.2 (RED) Patched maint/3.2 (GREEN)
3.9 N/A (requires-python >= 3.10) GUARDED (None)
3.10 UNGUARDED GUARDED (None)
3.11 UNGUARDED N/A (tested on master)
3.12 UNGUARDED N/A (tested on master)
3.13 UNGUARDED N/A (tested on master)
3.14 UNGUARDED N/A (tested on master)

Docker verification on Python 3.9 and 3.10 confirms both Test A (GC finalization) and Test B (atexit phase) return GUARDED on this branch.

Test plan

  • Full local test suite: 159 passed, 1 skipped, 0 failed (pytest, Python 3.10)
  • pylint: 10.00/10
  • TDD RED/GREEN verification via Docker (Python 3.9 + 3.10)
  • All 21 shutdown tests pass
  • Full CI on all supported Python versions

Relationship to other PRs

Backport of PR python-greenlet#499 (master) to maint/3.2 for greenlet 3.2.6, with all
shutdown guards made unconditional across Python 3.9-3.13.

The previous backport (3.2.5 / PR python-greenlet#495) only guarded Python < 3.11,
but the vulnerability exists on ALL Python versions: Py_IsFinalizing()
is set AFTER atexit handlers complete inside Py_FinalizeEx.

Two independent guards now protect all shutdown phases:

1. g_greenlet_shutting_down — atexit handler registered at module init
   (LIFO = runs first). Covers the atexit phase where
   Py_IsFinalizing() is still False.

2. Py_IsFinalizing() — covers the GC collection and later phases.
   A compatibility shim maps to _Py_IsFinalizing() on Python < 3.13.

These guards are checked in mod_getcurrent, PyGreenlet_GetCurrent,
GreenletChecker, MainGreenletExactChecker, ContextExactChecker,
clear_deleteme_list, ThreadState destructor,
_green_dealloc_kill_started_non_main_greenlet, and AddPendingCall.

Additional hardening:
- clear_deleteme_list() uses std::swap (zero-allocation)
- deleteme vector uses std::allocator (system malloc)
- ThreadState uses std::malloc/std::free
- clear_deleteme_list() preserves pending Python exceptions

TDD-certified: tests fail on greenlet 3.3.2 and pass with the fix
across Python 3.10-3.14. Docker verification on Python 3.9 and 3.10
confirms GUARDED on the maint/3.2 branch.

Also fixes:
- SPDX license identifier: Python-2.0 -> PSF-2.0
- test_dealloc_catches_GreenletExit_throws_other: use
  sys.unraisablehook for pytest compatibility
- test_version: skip gracefully on old setuptools
- Flaky USS memory test on Windows

Made-with: Cursor
@nbouvrette nbouvrette force-pushed the fix/safe-getcurrent-during-finalization-3.2 branch from bc4b8fb to 00feede Compare March 24, 2026 13:22
@nbouvrette nbouvrette changed the title Backport: Fix crash in getcurrent()/greenlet during early Py_FinalizeEx (3.2.6) Fix crashes during interpreter shutdown on all Python versions (3.2.6) Mar 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant