Skip to content

Test Discovery failing #24656

Closed
Closed

Description

Details: Currently test discovery will "hang" with version 2024.22.1 if you have >30 tests in your workspace due to a buffer overflow issue. This should only occur for Mac and Linux machines.

Workaround: Downgrade your python extension version to 2024.20.*.

Activity

  1. added
    bugIssue identified by VS Code Team member as probable bug
    regressionBug didn't exist in a previous release
    on Dec 24, 2024
  2. brianoflondon commented on Dec 24, 2024

    @brianoflondon

    If you don't have more than 30 tests in your workspace, are you even testing 🤪.

    Merry Christmas, Happy Holidays, thanks for all you do, we'll survive till you fix it.

    The way to downgrade: on the Extension page, click the drop down arrow on the Uninstall Button and you can "Install Specific Version...". Also make sure Auto Update is switched off or it will bounce you straight back to the broken one.

    2024.22.0 seems to be working for me.
    Image

  3. jflam commented on Dec 24, 2024

    @jflam

    Just ran into this bug this AM. Confirming that downgrading is the correct interim solution for this.

  4. bolderbrooklyn commented on Dec 24, 2024

    @bolderbrooklyn

    The codebase that I encountered this with and reported in #24655 is pretty small, only 16 tests and (per cov) 258 total statements. Downgrading to 2024.22.0, or disabling the pythonTestAdapter experiment in 2024.22.1, fixed the issue for me (but disabling the experiment also took away the neat little coverage indicators 😔, so I've chosen to downgrade for now).

  5. WellingtonNico commented on Dec 26, 2024

    @WellingtonNico

    Same problem here, there is not error logs, never ends finding tests, but in the python test logs pytest collect almost finished

    this is the image of the hanging

    Image

    this is the logs when it collects successfully

    [1m============================= test session starts ==============================�[0m
    platform darwin -- Python 3.11.3, pytest-8.3.4, pluggy-1.5.0
    rootdir: /Users/teste/projetos/pessoal/python/fastapi-todo-list
    configfile: pytest.ini
    plugins: asyncio-0.24.0, cov-6.0.0, anyio-4.7.0, xdist-3.6.1
    asyncio: mode=Mode.AUTO, default_loop_scope=None
    collected 1 item
    
    <Dir fastapi-todo-list>
      <Package tests>
        <Module test_first.py>
          <Function test_first_test>
    
    �[32m========================== �[32m1 test collected�[0m�[32m in 0.01s�[0m�[32m ===========================�[0m
    Starting now, all test run output will be sent to the Test Result panel, while test discovery output will be sent to the "Python" output channel instead of the "Python Test Log" channel. The "Python Test Log" channel will be deprecated within the next month. See https://github.com/microsoft/vscode-python/wiki/New-Method-for-Output-Handling-in-Python-Testing for details.

    this is the log for that project that does hang when collecting

    you can notice there is a log missing at the end that exists in the above log: test collected

    �[1m============================= test session starts ==============================�[0m
    platform darwin -- Python 3.12.8, pytest-8.3.4, pluggy-1.5.0 -- /Users/teste/projetos/pessoal/python/fastapi-todo_list/.venv/bin/python
    cachedir: .pytest_cache
    rootdir: /Users/teste/projetos/pessoal/python/fastapi-todo_list
    configfile: pytest.ini
    plugins: asyncio-0.24.0, cov-6.0.0, anyio-4.7.0, xdist-3.6.1
    asyncio: mode=Mode.AUTO, default_loop_scope=None
    �[1mcollecting ... �[0mcollected 17 items
    
    <Dir fastapi-todo_list>
      <Package tests>
        <Dir auth>
          <Module test_auth_api.py>
            <Coroutine test_signup>
            <Coroutine test_login>
            <Coroutine test_me_with_todos>
            <Coroutine test_me_without_todos>
            <Coroutine test_logout_on_get_and_post>
        <Dir main>
          <Module test_main.py>
            <Coroutine test_healthy>
        <Dir todo>
          <Module test_todo_api.py>
            <Coroutine test_get_user_todos_security>
            <Coroutine test_get_user_todos>
            <Coroutine test_create_todo_security>
            <Coroutine test_create_todo>
            <Coroutine test_create_todo_title_validation>
            <Coroutine test_update_todo_security>
            <Coroutine test_update_todo>
            <Coroutine test_delete_todo_security>
            <Coroutine test_delete_todo>
            <Coroutine test_count_user_todos>
            <Coroutine test_get_todos_full>

    I tried both with same python version and it does not work as well, both projects have the same dependencies, the only difference are the tests. In my Django projects the tests stopped working too, they were working just fine yesterday(2024/12/23), the same problem occurs, hangs when collecting the tests.

    what I already tried:

    • fresh install
    • new user with totally fresh install and only the needed extensions(I thought it was gonna work)
    • different python versions

    I'm currently using macOS, latest version with m3 pro chip, I also made sure I'm using the correct build version for apple silicon.

    update

    inside devcontainer it finds the tests and also run, but I was not able to debug them like yesterday, the debug menu appears, I can even set the breakpoints but the code never pauses(and yes, I'm did debug tests inside dev container before and it was working).
    I have just a few extensions inside dev container, more than when I tested with fresh instalation with new user, for me it looks like something maybe related to macos

    what worked

    I downgraded python, and debugpy extension version, the tests were found, for all projects, but I still was not able to debug them, accidentaly I noticed it worked for a new project that I created just to test this case, so I cloned my repo into a new folder and everything worked again, so I also tried to update the python extensions again and It broke again, :(

  6. patrickkidd commented on Dec 26, 2024

    @patrickkidd

    I also had this issue with 2024.22.1 - same symptoms and output panel contents - and downgrading to 2024.22.0 fixes it for me. I am reproducing it by adding additional test cases or longer test case names like the following snippet which suggests a buffer overflow to me. Sometimes it freezes at different buffer lengths but usually within 100 chars of the example in the snippet:

    def test_one():
        pass
    
    def test_two():
        pass
    
    def test_three():
        pass
    
    def test_four():
        pass
    
    def test_five():
        pass
    
    def test_six():
        pass
    
    def test_seven():
        pass
    
    def test_eight():
        pass
    
    def test_nine():
        pass
    
    def test_ten():
        pass
    
    def test_eleven():
        pass
    
    def test_twelve():
        pass
    
    def test_thirteen():
        pass
    
    def test_fourteen():
        pass
    
    def test_fifteen():
        pass
    
    def test_sixteen():
        pass
    
    def test_seventeen():
        pass
    
    def test_eighteen():
        pass
    
    def test_nineteen():
        pass
    
    def test_twenty():
        pass
    
    def test_twenty_one():
        pass
    
    def test_twenty_two():
        pass
    
    def test_twenty_three():
        pass
    
    def test_twenty_four():
        pass
    
    def test_twenty_five():
        pass
    
    def test_twenty_six():
        pass
    
    ## Uncomment only one of the below examples at a time to reproduce the problem, at least on my machine.
    
    ## SUCCEEDS
    
    # def test_twenty_seven():
    #     pass
    
    
    ## SUCCEEDS
    
    # def test_twenty_1234567891234567890123456789012345():
    #     pass
    
    ## FREEZES
    
    # def test_twenty_12345678912345678901234567890123456():
    #     pass
    
    
    ## FREEZES
    
    # def test_twenty_seven():
    #     pass
    
    # def test_twenty_nine():
    #     pass
    
    
    
  7. BobKerns commented on Dec 27, 2024

    @BobKerns

    Probably redundant since it's been identified as a buffer overflow issue, but the specific place it hangs in the plugin is line 943 in __init__.py :

                request = f"""content-length: {len(data)}\ncontent-type: application/json\n\n{data}"""
                __writer.write(request) # HANGS HERE 😞
    
                __writer.flush()

    This may be a repeat bug? I've had discovery suddenly start hanging like this in the past, but didn't suspect an update, and never got to track it this far. It may be worth a warning in the code on the other end, and a test case.

  8. BobKerns commented on Dec 27, 2024

    @BobKerns

    Maybe one symptom of a larger problem? I'm seeing similar hangs elsewhere:

    • Markdown Preview
    • Markdown Preview Enhanced
    • Source Control

    Source Control involves a large number of pending changes.
    Markdown fails even with a small .md file, but perhaps prepended CSS, etc. makes it start off large?

  9. 30 remaining items

  10. brianoflondon commented on Jan 9, 2025

    @brianoflondon

    2024.23.2025010801 is working for me too.

  11. eleanorjboyd commented on Jan 9, 2025

    @eleanorjboyd
    Author

    As an update the point release 2024.22.2 is out and has this fix for those who are not on the pre-release. Thanks!

  12. bolderbrooklyn commented on Jan 10, 2025

    @bolderbrooklyn

    Eleanor Boyd (@eleanorjboyd) can confirm that this is fixed for me and fixes #24655 as well, thank you!

  13. PaleNeutron commented on Feb 5, 2025

    @PaleNeutron

    I have switched to 2024.23.2025012801 and this problem is still exist. Test discovering hangs and no any log for it after

    2025-02-05 14:02:42.284 [debug] Testing: Refreshing all test data
    2025-02-05 14:02:42.287 [info] Discover tests for workspace name: sqlmodel - uri: /home/john/contribute/sqlmodel
    2025-02-05 14:02:42.293 [debug] Starting Test Discovery named pipe
    
  14. WellingtonNico commented on Feb 5, 2025

    @WellingtonNico

    I have switched to 2024.23.2025012801 and this problem is still exist. Test discovering hangs and no any log for it after

    2025-02-05 14:02:42.284 [debug] Testing: Refreshing all test data
    2025-02-05 14:02:42.287 [info] Discover tests for workspace name: sqlmodel - uri: /home/john/contribute/sqlmodel
    2025-02-05 14:02:42.293 [debug] Starting Test Discovery named pipe
    

    Try to clone your project to a new folder in a different directory and try again

  15. eleanorjboyd commented on Feb 6, 2025

    @eleanorjboyd
    Author

    Hi John Lyu (@PaleNeutron) if you are still seeing this bug- please open a new issue and I can help you out there. Thanks

  16. cmiller73084 commented on Jul 2, 2025

    @cmiller73084

    I'm running into this (or a extremely similar) issue. code-server: 4.100.3, Code 1.100.3. Python extension: 2025.4.0.

    Test discovery hangs with this in the output log

    2025-07-02 14:49:58.649 [debug] Testing: Refreshing all test data
    2025-07-02 14:49:58.650 [debug] Found cached env for /home/cwmiller/pytest_vscode/.venv/bin/python
    2025-07-02 14:49:58.650 [info] Discover tests for workspace name: pytest_vscode - uri: /home/cwmiller/pytest_vscode
    2025-07-02 14:49:58.650 [debug] Found cached env for /home/cwmiller/pytest_vscode/.venv/bin/python
    2025-07-02 14:49:58.653 [debug] Starting Test Discovery named pipe
    2025-07-02 14:50:15.287 [debug] Found cached env for ...

    I can get things to work by downgrading the Python extension to 2024.22.1.

    I can reproduce the issue with a fresh install of VSCode server running against a brand new workspace directory.

  17. hhalaby commented on Feb 10, 2026

    @hhalaby

    Anyone landing here from google, there's seems to be a new issue with systemd services failing and causing this error again. Here's the solution on StackOverFlow

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

area-testingbugIssue identified by VS Code Team member as probable bugregressionBug didn't exist in a previous releasetriage-neededNeeds assignment to the proper sub-team

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions