Per-interpreter runtime state and a process-wide GC stop-the-world - #8517

Groundwork for multiple interpreters (PEP 734 / PEP 684), plus a GC data race that
having more than one interpreter exposes. No Python-facing API is added yet:
sys.implementation.supports_isolated_interpreters stays false and there is no
_interpreters module in this PR.

Layering

The types line up with CPython so the stdlib module can be added on top later:

What changed

Per-interpreter state. PyGlobalState gains interpreter_id / whence /
is_main. Interpreter::create_subinterpreter() builds an interpreter that shares
the process-wide Context (immortal builtin types, safe because builtin types are
IMMUTABLETYPE) but gets its own PyGlobalState, sys.modules, builtins module,
codec registry, warnings state, thread registry and stop-the-world state. Config,
module defs and frozen modules are cloned from the parent.

Interpreter registry (vm/runtime.rs): monotonic ids, weak entries so the
registry never keeps an interpreter alive, whence tracking, and a main_id
recorded from the first is_main interpreter for a future _interpreters.get_main().
There is also a runtime-owned table (store_owned_interpreter / take_owned_interpreter),
which is the ownership anchor _interpreters.create() will need, since Python
receives only an id. It is threading-gated because Interpreter: Send only holds
for Arc-backed builds.

Per-interpreter thread slots. CPython keeps a PyThreadState per
(thread, interpreter) pair; INTERP_THREAD_SLOTS mirrors that, so
sys._current_frames() and stop-the-world are scoped to one interpreter.

Signals stay owned by the main interpreter: subinterpreters no longer reinstall
SIGINT handlers, and _thread._is_main_interpreter() now reflects the caller.

Registry lifetime. An interpreter is registered before initialize() runs any
bytecode, is released when its last PyRc<PyGlobalState> goes away rather than when
the Interpreter handle drops (workers from new_thread() outlive the handle), and
every interpreter — not just the forking one — is repaired in a forked child, since
the collector now stops all of them.

GC. The cyclic collector's generation lists are process-global, so a collection
reads and frees objects owned by every interpreter — but CollectStopTheWorld
stopped only the collecting interpreter, leaving other interpreters' threads free to
mutate the same object graph during the reference-subtraction, reachability and
snapshot phases. It now stops every live interpreter in registry id order and
restarts in reverse. The global collecting mutex serializes collectors
process-wide and fork acquires a single interpreter's exclusion, so the exclusion
orders cannot cycle. StopTheWorldState methods now take &PyGlobalState instead
of &VirtualMachine, since an interpreter's world must be stoppable without
holding a VM for it — that is an API change for embedders calling these directly.

Interpreter hot paths

The interpreter-loop work stacked on top of the foundation, measured in instructions
retired per operation against 57428cd92 (the last commit before it) so the numbers
do not move with machine load:

Frames for a simple call now come from the thread's data stack, cleared frame blocks
are reused without re-zeroing, the per-instruction safepoint and the call preamble do
less work per instruction, an attribute lookup reaches the instance dict without
cloning it, a call builds its argument vector once, dict lookups and iteration steps
take a single read guard, and the GC tracking counters moved off the write barrier.

Frame lifetime

Data stack frames diverged from the reference implementation in ways that showed up as
extra references and, across threads, as reads of a buffer the owning thread was still
writing:

• Materializing a running frame copied its fast locals into the frame object, so a
  single traceback entry gave every local in that frame an extra reference. The copy
  is now empty and exit_iframe fills it when the frame returns, which is what the
  chain-materializing path already did.
• The tail call trampoline held the callee's function object until the caller was
  re-entered; it is dropped when the suspended caller is popped.
• sys._current_frames(), _thread._current_exceptions() and the cross-thread
  f_back path handed back frame objects still linked to the running frames, so the
  owner wrote its locals into the same buffers the reader was cloning out of. They now
  take a detached copy under stop-the-world. Reading a live foreign frame's locals
  through any other path raises instead, gated on a new attached_tid.
• set_current_frame published the Py<FrameObject> address in a slot whose readers
  pass it to Py::from_payload_ptr, which subtracts the object header — a SIGSEGV on
  every cross-thread frame read, present before this branch.
• Data stack frame reuse compared alignment-rounded sizes, so a frame up to 15 bytes
  larger than the previous one could skip zeroing slots that were never cleared.

Known limitations

• The collector is process-wide, not per-interpreter. gc.disable(), thresholds,
  gc.garbage and gc.get_objects() all observe process-wide state. Making it
  per-interpreter additionally requires routing untrack_object — called from
  default_dealloc, where no VM is in scope — to the owning interpreter's lists.
  Documented on gc_state().
• Entering a second interpreter from inside another interpreter's enter() on the
  same OS thread is not supported yet. enter_vm only attaches at the outermost
  section, so a nested cross-interpreter enter would run with the inner slot
  DETACHED. Nothing in tree does this; the _PyThreadState_Swap equivalent belongs
  with the _interpreters work that needs it.
• Subclassing a shared builtin type from a subinterpreter registers the subclass on
  the shared type, so it is visible through __subclasses__ in other interpreters.

Tests

cargo test -p rustpython-vm --features threading and the default (non-threading)
build both pass, as do clippy and -m test for test_gc, test_threading and
test_fork1.

New tests cover interpreter identity and registration, module/builtins isolation,
subinterpreter creation while the parent is entered, concurrent and overlapping
execution across interpreters, and runtime-owned interpreter lifecycle. The two GC
tests were each checked to fail without their fix:
stop_the_world_parks_threads_of_another_interpreter asserts a thread entered in one
interpreter can park another interpreter's threads.

A full -m test sweep leaves two failures on macOS, test.test_future_stmt.test_future
(a REPL syntax-error assertion) and test_pyrepl; both fail the same way on the commit
before the frame work. Reference counts for locals and for a tail call's function object were compared
against CPython 3.14 rather than eyeballed, and the cross-thread paths were run
repeatedly under a spinning worker — they used to crash roughly half the time.

Summary by CodeRabbit

• New Features

  • Added support for creating and managing isolated subinterpreters.
  • Added interpreter identity and lifecycle information.
  • Interpreter support is now accurately reported through sys.implementation.
  • Improved isolation for modules, configuration, threads, runtime state, and garbage collection.
  • Added interpreter-specific object visibility and garbage collection controls.

• Bug Fixes

  • Corrected signal-handler setup so only the main interpreter installs process-level handlers.
  • Improved garbage collection, traceback, frame inspection, forking, and thread coordination across interpreters.
  • Fixed interpreter-specific thread-state handling during nested execution and cleanup.
  • Improved blocking queue and I/O operations during thread coordination.