Skip to content

fix: isolate gathered task results by result backend - #728

Open
qyf700 wants to merge 2 commits into
taskiq-python:masterfrom
qyf700:fix/gather-backend-isolation
Open

qyf700 wants to merge 2 commits into
taskiq-python:masterfrom
qyf700:fix/gather-backend-isolation

Conversation

@qyf700

@qyf700 qyf700 commented Nov 18, 2026

Copy link
Copy Markdown

Problem

gather() keys its pending set and result map only by task_id, although each AsyncTaskiqTask also carries its own result backend. When two independent backends contain the same custom task ID, one branch's result is returned for both positions. If only one backend has a result, gather() can report success while the other branch is still pending.

This applies to explicit IDs reused in independent result stores, not random UUID collisions. It reproduces on upstream 1818f5381978eeea5825b7d32397fe8950f5eae3.

Reproduction

The following runs without a worker service, network access, model, or credentials:

import asyncio

from taskiq import gather
from taskiq.brokers.inmemory_broker import InmemoryResultBackend
from taskiq.result import TaskiqResult
from taskiq.task import AsyncTaskiqTask


async def main():
    left, right = InmemoryResultBackend(), InmemoryResultBackend()
    for backend, value in ((left, "research result"), (right, "review result")):
        await backend.set_result(
            "agent-run-42",
            TaskiqResult(is_err=False, return_value=value, execution_time=0),
        )
    results = await gather(
        AsyncTaskiqTask("agent-run-42", left),
        AsyncTaskiqTask("agent-run-42", right),
        periodicity=0,
    )
    print([result.return_value for result in results])


asyncio.run(main())

Before: ['research result', 'research result']

After: ['research result', 'review result']

For the premature-completion case, store only the left result and pass timeout=0.01 to gather(). Before this change it still returns the duplicated successful result; afterwards it waits for the missing branch and raises TaskiqResultTimeoutError. The tests also reproduce the wrong-result case through two real InMemoryBroker instances and the public task/kicker APIs.

Fix

Key pending work and fetched results by (id(task.result_backend), task.task_id). This preserves input ordering and fetches equivalent handles sharing the same backend instance only once, including backends that consume a result on read. The function signature, polling interval, and timeout clock are unchanged. The docstring now describes the identity contract.

Backend identity is the Python object, not a physical store inferred from connection settings. Separate backend objects connected to the same physical store are treated independently; callers needing a single consuming read should share the backend instance.

Validation

Python 3.12.16, macOS arm64, repository lockfile dependencies:

  • Final six regression cases against unchanged upstream: 5 failed, 1 passed.
  • Full suite after the fix: 405 passed (python -m pytest -q).
  • Ruff passed for both changed Python files.
  • Black check passed for both changed files, targeting Python 3.10.
  • mypy taskiq tests: no issues in 147 source files.
  • The published files were compared byte-for-byte with the tested local files.

Coverage includes both input orders, either backend remaining pending, a result published after the initial poll, and duplicate handles with a consuming backend. No live Redis or model service was tested.

Related work

Rechecked open and closed gather PRs before submission. #711 addresses the final polling delay and #705 addresses timeout clock measurement; their patches do not address backend-scoped result identity. This change leaves those behaviors unchanged.

@codecov

codecov Bot commented Nov 18, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.14%. Comparing base (1818f53) to head (41e649e).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #728      +/-   ##
==========================================
+ Coverage   86.06%   86.14%   +0.07%     
==========================================
  Files          69       69              
  Lines        2778     2779       +1     
==========================================
+ Hits         2391     2394       +3     
+ Misses        387      385       -2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants