Skip to content

tests: suppress data_too_large in all fuzz tests that draw in a loop - #380

Open
pylaterreur wants to merge 1 commit into
indygreg:mainfrom
pylaterreur:tests-suppress-data-too-large
Open

pylaterreur wants to merge 1 commit into
indygreg:mainfrom
pylaterreur:tests-suppress-data-too-large

Conversation

@pylaterreur

Copy link
Copy Markdown

The fuzz tests that draw read, write or chunk sizes from data() until their input is consumed can overrun Hypothesis's buffer when they sample a large input, and the data_too_large health check then fails them depending on the seed:

hypothesis.errors.FailedHealthCheck: Examples routinely exceeded the max allowable size. (20 examples overran while generating 8 valid ones).

This was suppressed for one test at a time when it tripped (e98b641, 3ea96b6). In the last 60 scheduled runs of test.yml on main it failed:

  • TestCompressor_compressobj_fuzzing::test_flush_block: ubuntu-24.04-arm, 3.14, C backend (2026-11-08);
  • TestCompressor_compressobj_fuzzing::test_random_input_sizes: macos-26, 3.12, CFFI backend (2026-10-31), and windows-11-arm, CFFI backend (2026-11-04);
  • and on a fork, TestCompressor_chunker_fuzzing::test_flush_block (windows-2022, 3.12, x86).

This suppresses it in the 21 such tests that did not suppress it yet. test_read_across_frames_false had no settings, so it gets a suppress_health_check list with just this check.

With the seeds those CI runs printed, both compressobj tests fail the health check locally on main (Linux, 3.12, HYPOTHESIS_PROFILE=ci, ZSTD_SLOW_TESTS=1) and pass with this change:

pytest "tests/test_compressor_fuzzing.py::TestCompressor_compressobj_fuzzing::test_flush_block" --hypothesis-seed=72514116356851535314188617373821141879
PYTHON_ZSTANDARD_IMPORT_POLICY=cffi pytest "tests/test_compressor_fuzzing.py::TestCompressor_compressobj_fuzzing::test_random_input_sizes" --hypothesis-seed=208916250311990308935113999347445447953

tests/test_compressor_fuzzing.py and tests/test_decompressor_fuzzing.py pass with the ci profile: 41 tests with the C backend, 39 passed and 2 skipped with the CFFI backend. ruff check passes and ruff format leaves the files unchanged.

Prepared with the help of Claude Code.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GwUvhmzgfkiZ6vo4uGyesN

The fuzz tests that draw read, write or chunk sizes from data() until
their input is consumed overrun Hypothesis's buffer on large inputs, and
the data_too_large health check fails them depending on the seed. This
was patched for one test at a time (e98b641, 3ea96b6). In the last 60
scheduled runs of test.yml on main, it failed
TestCompressor_compressobj_fuzzing::test_flush_block (ubuntu-24.04-arm,
3.14, 2026-11-08) and ::test_random_input_sizes (macos-26, 3.12,
2026-10-31; windows-11-arm, 2026-11-04), and on a fork run
TestCompressor_chunker_fuzzing::test_flush_block.

Suppress it in the 21 such tests that did not yet. With the seeds of the
2026-11-08 and 2026-10-31 runs, the two compressobj tests fail the
health check locally before this change and pass after it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GwUvhmzgfkiZ6vo4uGyesN
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant