Skip to content

fix: start and stop MultiBroker result backend once - #727

Open
zakharkiv-vitalik wants to merge 1 commit into
taskiq-python:masterfrom
zakharkiv-vitalik:fix/multibroker-shared-result-backend
Open

zakharkiv-vitalik wants to merge 1 commit into
taskiq-python:masterfrom
zakharkiv-vitalik:fix/multibroker-shared-result-backend

Conversation

@zakharkiv-vitalik

Copy link
Copy Markdown
Contributor

Fixes #726

Problem

MultiBroker.with_result_backend() assigned the same backend instance to every inner broker. Since every AsyncBroker starts and stops its own result_backend, a shared backend got N+1 startup() / shutdown() calls for N inner brokers: resources opened in startup() (e.g. connection pools) leaked and repeated shutdown() could fail. See #726 for a reproduction.

Solution

The propagation was needed in the first version of #723, when tasks were registered on inner brokers. After the review (comment) tasks are registered only on the MultiBroker, so the kicker, the receiver and the progress tracker always use its backend, and inner brokers never read theirs.

  • dropped the with_result_backend() override, the base AsyncBroker implementation sets the backend on the MultiBroker only;
  • updated the docs note about result backends.

Tests

  • test_shared_result_backend_started_once / test_shared_result_backend_shut_down_once — fail on master with 3 == 1, pass now;
  • test_with_result_backend_propagates_to_inner_brokers → test_with_result_backend_is_not_shared_with_inner_brokers;
  • the existing test_receiver_processes_tasks_from_all_brokers (kiq() → worker → wait_result()) still passes without propagation.

pytest (401 passed), ruff, black, mypy, typos pass locally.

@codecov

codecov Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.05%. Comparing base (1818f53) to head (eb939c8).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #727      +/-   ##
==========================================
- Coverage   86.06%   86.05%   -0.01%     
==========================================
  Files          69       69              
  Lines        2778     2769       -9     
==========================================
- Hits         2391     2383       -8     
+ Misses        387      386       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

MultiBroker.with_result_backend() assigned the same backend to every
inner broker, so each of them started and stopped it too: N+1 calls
for N inner brokers, leaking resources opened in startup() and failing
on repeated shutdown().

Tasks are registered only on the MultiBroker, so kicks, results and
progress always use its backend and inner brokers never need one.
Drop the override and keep the backend on the MultiBroker only.
@zakharkiv-vitalik
zakharkiv-vitalik force-pushed the fix/multibroker-shared-result-backend branch from 392d057 to eb939c8 Compare November 18, 2026 06:52

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MultiBroker starts and shuts down a shared result backend N+1 times

3 participants