Repository navigation
Conversation
…ackends Add two resources that backends share, after TensorStore's context resources: ThreadPool runs blocking work, and Limiter caps operations in flight for threads and tasks on any event loop in one first-come, first-served queue. Pass one to several backends to bound them together; left out, a backend uses a process-wide default (ThreadPool.default(), max(4, cpu_count) threads; one Limiter(32) per obstore store class). LocalBackend takes `threads` and runs async operations and batches on it; closing the backend no longer shuts a pool down, since whoever creates a pool owns it. ObjectStoreBackend takes `concurrency`, a Limiter, and holds a slot for every operation, sync and async, single and batched, instead of bounding each async batch on its own. A slot is re-entrant within one operation, so an operation that calls another can't wait on itself. A batch started on one of its pool's own threads runs on that thread rather than wait on the pool. Pickling keeps resources shared: pickle stores each object once, and defaults unpickle as the receiving process's defaults. BREAKING CHANGE: LocalBackend's `concurrency: int` is now `threads: ThreadPool`, and ObjectStoreBackend's `concurrency` is a Limiter instead of an int. LocalBackend.aclose is gone. Assisted-by: ClaudeCode:claude-opus-5-5
Assisted-by: ClaudeCode:claude-opus-5-5
Resources move from backends to a Context that every store view carries,
as in TensorStore. A Context holds a ThreadPool and a Limiter per
resource key, is immutable, and resolves keys through its parents to
process-wide defaults. Backends hold no resources: each names the
`resource_key` its operations are limited under ("local", "memory",
"obstore.<StoreClass>"), and the store gates every operation with the
context's limiter for that key.
Store and AsyncStore take `context=`, and gain `with_context` and
`limited(n)`, a view whose operations together run at most n at once.
`Limiter(n, within=parent)` holds a slot of both, so a scoped limit
never escapes the one it narrows (where TensorStore's child contexts
replace a resource, this nests). Slots are taken innermost first, and
where the work runs: AsyncStore runs a sync-only backend on the
context's thread pool and takes the slot on the pool thread, so no
operation holds a slot while it waits for a thread.
LocalBackend is now sync-only; AsyncStore adapts it. Batches run single
operations concurrently under the context (a Store batch on the pool, an
AsyncStore batch as tasks, up to the limiter's limit), which also makes
sync obstore batches concurrent. Native batch methods receive the
context. Slots are no longer re-entrant: re-entrancy through a context
variable leaked into tasks and, on free-threaded 3.14, into new threads,
letting a fan-out run unbounded under one slot.
BREAKING CHANGE: LocalBackend's `threads`, its async methods and
ObjectStoreBackend's `concurrency` are gone; pass resources in a
Context. Backends must provide `resource_key`. Batch protocol methods
take the context.
Assisted-by: ClaudeCode:claude-opus-5-5
The cache is a `CachePool` in the store's context, off by default. Its bytes live in a tier, which is any writable store, so range reads, batches and the conformance suite come from the tier. An in-memory index holds what a store can't: source generations, ages, absences and eviction order. Assisted-by: ClaudeCode:claude-opus-5-5
Assisted-by: ClaudeCode:claude-opus-5-5
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤖 AI text below 🤖
Add
Context, the resources azarr-storagestore's operations draw on: aThreadPooland aLimiterper backend resource key, with process-wide defaults. Every store view carries a context (Store(backend, context=...),with_context), stores sharing a context share its limits, andstore.limited(n)bounds a burst of operations within the context's limit.AsyncStorenow runs a sync-only backend on the context's thread pool, and batches run concurrently under the context.Design
Resource limits are part of a store's context, after TensorStore's
Context(context.h,concurrency_resource.h,s3_resource.h). zarrs has no I/O limits of its own (resources.rscarries only a compute target); itsSyncToAsyncStorageAdaptertakes an injectedspawn_blocking, which here is the context's pool.Context(threads=..., limits={key: Limiter}): immutable;child()overrides some resources and inherits the rest; keys left out resolve to process-wide defaults (ThreadPool.default()ofmax(4, cpu_count)threads; oneLimiter(32)per key, TensorStore'ss3_request_concurrencydefault).resource_key("local","memory","obstore.S3Store", ...), and every operation through a view holds a slot ofview.context.limiter(key). Stores sharing a context share its limits; every S3 backend in a process shares one limit by default. A context is not part of a store's identity.store.limited(n)andstore.with_context(ctx)are views.limitedusesLimiter(n, within=parent), which holds a slot of both, so a scoped limit never escapes the one it narrows; TensorStore's child contexts replace a resource instead, which would let it escape.AsyncStoreruns a sync-only backend on the context's pool and takes the slot on the pool thread, so no operation holds a slot while it waits for a thread.LocalBackendis sync only now;AsyncStoreadapts any sync backend. Batches run single operations concurrently under the context (on the pool forStore, as tasks forAsyncStore, up to the limiter's limit), which also makes sync obstore batches concurrent. Native batch methods receive the context.gather's tasks, which let a fan-out run unbounded under one slot. Since stores take exactly one slot per operation, nothing needs re-entrancy.Testing
tests/test_resources.py: the limiter's bound across threads and two event loops; nested limiters bounded by both; innermost-first ordering; first-come, first-served order; cancellation before and after a grant, and between nested slots; context resolution through parents to defaults; views carrying their context; pickled sharing; sync and async operations on a native-async and a sync-only backend waiting for the context's limiter; a limited view bounding its batches; a deterministic one-thread, one-slot scenario where taking the slot before getting a thread would deadlock; a batch started on a pool thread running on that thread; one test per bad argument.python:3.12-slim); pyright strict and ruff are clean.Stack
Base: #460 (
feat/zarr-storage-api), which targets the topic branchtopic/zarr-storage. Retarget this PR totopic/zarr-storageonce #460 merges there. The first commit put resources on backends; the second moves them into the store's context, as asked in review; squash them when merging.🤖 Generated with Claude Code