Skip to content

feat(zarr-storage)!: make resource limits part of the store's context - #464

Draft
d-v-b wants to merge 5 commits into
feat/zarr-storage-apifrom
feat/zarr-storage-resources
Draft

d-v-b wants to merge 5 commits into
feat/zarr-storage-apifrom
feat/zarr-storage-resources

Conversation

@d-v-b

@d-v-b d-v-b commented Nov 18, 2026 •

Copy link
Copy Markdown
Owner

🤖 AI text below 🤖

Add Context, the resources a zarr-storage store's operations draw on: a ThreadPool and a Limiter per backend resource key, with process-wide defaults. Every store view carries a context (Store(backend, context=...), with_context), stores sharing a context share its limits, and store.limited(n) bounds a burst of operations within the context's limit. AsyncStore now runs a sync-only backend on the context's thread pool, and batches run concurrently under the context.

Design

Resource limits are part of a store's context, after TensorStore's Context (context.h, concurrency_resource.h, s3_resource.h). zarrs has no I/O limits of its own (resources.rs carries only a compute target); its SyncToAsyncStorageAdapter takes an injected spawn_blocking, which here is the context's pool.

  • Context(threads=..., limits={key: Limiter}): immutable; child() overrides some resources and inherits the rest; keys left out resolve to process-wide defaults (ThreadPool.default() of max(4, cpu_count) threads; one Limiter(32) per key, TensorStore's s3_request_concurrency default).
  • Backends hold no resources. Each names a resource_key ("local", "memory", "obstore.S3Store", ...), and every operation through a view holds a slot of view.context.limiter(key). Stores sharing a context share its limits; every S3 backend in a process shares one limit by default. A context is not part of a store's identity.
  • store.limited(n) and store.with_context(ctx) are views. limited uses Limiter(n, within=parent), which holds a slot of both, so a scoped limit never escapes the one it narrows; TensorStore's child contexts replace a resource instead, which would let it escape.
  • Slots are taken innermost first (a burst queued on a narrow limit holds none of the wide one) and where the work runs: AsyncStore runs a sync-only backend on the context's pool and takes the slot on the pool thread, so no operation holds a slot while it waits for a thread.
  • LocalBackend is sync only now; AsyncStore adapts any sync backend. Batches run single operations concurrently under the context (on the pool for Store, as tasks for AsyncStore, up to the limiter's limit), which also makes sync obstore batches concurrent. Native batch methods receive the context.
  • Slots are not re-entrant. A first version made them re-entrant through a context variable; free-threaded Python 3.14 exposed that new threads inherit the context, and so do gather's tasks, which let a fan-out run unbounded under one slot. Since stores take exactly one slot per operation, nothing needs re-entrancy.
  • A batch started on one of its pool's own threads runs on that thread instead of waiting for the pool. Pickling keeps sharing, and defaults unpickle as the receiving process's defaults.
Testing
  • tests/test_resources.py: the limiter's bound across threads and two event loops; nested limiters bounded by both; innermost-first ordering; first-come, first-served order; cancellation before and after a grant, and between nested slots; context resolution through parents to defaults; views carrying their context; pickled sharing; sync and async operations on a native-async and a sync-only backend waiting for the context's limiter; a limited view bounding its batches; a deterministic one-thread, one-slot scenario where taking the slot before getting a thread would deadlock; a batch started on a pool thread running on that thread; one test per bad argument.
  • The agent mutation-tested the design: holding the slot while waiting for a thread, taking slots outermost first, dropping the submitter's context in pool threads (before re-entrancy was removed), not gating sync operations, and removing the pool-thread check each fail a test; deadlock-prone tests run under timeouts.
  • 1766 tests pass on Python 3.11 and free-threaded 3.14 (macOS) and in a Linux container (python:3.12-slim); pyright strict and ruff are clean.
Stack

Base: #460 (feat/zarr-storage-api), which targets the topic branch topic/zarr-storage. Retarget this PR to topic/zarr-storage once #460 merges there. The first commit put resources on backends; the second moves them into the store's context, as asked in review; squash them when merging.

🤖 Generated with Claude Code

…ackends

Add two resources that backends share, after TensorStore's context
resources: ThreadPool runs blocking work, and Limiter caps operations in
flight for threads and tasks on any event loop in one first-come,
first-served queue. Pass one to several backends to bound them together;
left out, a backend uses a process-wide default (ThreadPool.default(),
max(4, cpu_count) threads; one Limiter(32) per obstore store class).

LocalBackend takes `threads` and runs async operations and batches on
it; closing the backend no longer shuts a pool down, since whoever
creates a pool owns it. ObjectStoreBackend takes `concurrency`, a
Limiter, and holds a slot for every operation, sync and async, single
and batched, instead of bounding each async batch on its own. A slot is
re-entrant within one operation, so an operation that calls another
can't wait on itself. A batch started on one of its pool's own threads
runs on that thread rather than wait on the pool. Pickling keeps
resources shared: pickle stores each object once, and defaults unpickle
as the receiving process's defaults.

BREAKING CHANGE: LocalBackend's `concurrency: int` is now
`threads: ThreadPool`, and ObjectStoreBackend's `concurrency` is a
Limiter instead of an int. LocalBackend.aclose is gone.

Assisted-by: ClaudeCode:claude-opus-5-5
Resources move from backends to a Context that every store view carries,
as in TensorStore. A Context holds a ThreadPool and a Limiter per
resource key, is immutable, and resolves keys through its parents to
process-wide defaults. Backends hold no resources: each names the
`resource_key` its operations are limited under ("local", "memory",
"obstore.<StoreClass>"), and the store gates every operation with the
context's limiter for that key.

Store and AsyncStore take `context=`, and gain `with_context` and
`limited(n)`, a view whose operations together run at most n at once.
`Limiter(n, within=parent)` holds a slot of both, so a scoped limit
never escapes the one it narrows (where TensorStore's child contexts
replace a resource, this nests). Slots are taken innermost first, and
where the work runs: AsyncStore runs a sync-only backend on the
context's thread pool and takes the slot on the pool thread, so no
operation holds a slot while it waits for a thread.

LocalBackend is now sync-only; AsyncStore adapts it. Batches run single
operations concurrently under the context (a Store batch on the pool, an
AsyncStore batch as tasks, up to the limiter's limit), which also makes
sync obstore batches concurrent. Native batch methods receive the
context. Slots are no longer re-entrant: re-entrancy through a context
variable leaked into tasks and, on free-threaded 3.14, into new threads,
letting a fan-out run unbounded under one slot.

BREAKING CHANGE: LocalBackend's `threads`, its async methods and
ObjectStoreBackend's `concurrency` are gone; pass resources in a
Context. Backends must provide `resource_key`. Batch protocol methods
take the context.

Assisted-by: ClaudeCode:claude-opus-5-5
@d-v-b d-v-b changed the title feat(zarr-storage)!: share thread pools and operation limits across backends feat(zarr-storage)!: make resource limits part of the store's context Nov 18, 2026
The cache is a `CachePool` in the store's context, off by default. Its bytes
live in a tier, which is any writable store, so range reads, batches and the
conformance suite come from the tier. An in-memory index holds what a store
can't: source generations, ages, absences and eviction order.

Assisted-by: ClaudeCode:claude-opus-5-5

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant