Repository navigation
High Impact Suggestions for the Team #21
Description
Hi Test Runner Team,
I realize that everyone here is a volunteer and contributes whatever they want (or doesn't even contribute at all). There are some issues that I think may be flying under your radar that I think could have a big impact on the test runner experience:
- Finish/stabilize code coverage. I've seen this in use a fair amount in the wild, but people always point out that it is still experimental. I think it's worth evaluating the accuracy and considering stabilizing it. The performance would also be good to look at, but should not be considered a blocker since it can be improved without impacting users. Coverage also does not support statement level coverage, but this should not be a blocker because:
- This is an artifact of V8's coverage output. Outside of changing V8, getting statement level coverage would require an additional parsing pass over all of the source code to map statements to positions in files. This would almost certainly be bad for performance.
- I believe C8 also does not support statement coverage for V8 output. My understanding is that line coverage is reported as statement coverage because it's good enough.
- It can always be added in a semver minor change if someone comes up with a good solution.
- Combining test runner flags with npm scripts (--test-name-pattern needing to come before filenames is hostile to npm scripts node#51384) - There are some possible solutions described in that thread. Right now, it's possible to work around this with
NODE_OPTIONS, but it's not really a nice solution. That issue appears to be a blocker to jsdom adopting the test runner for example. This would be a massive DX improvement. - Properly associating
console.log()output with its test in the reported output (node:testcustom reporters gettest:stdoutandtest:stderrevents beforetest:dequeuenode#53103) - One way to solve this could be to add code toConsole, similar to what diagnostic channels do. Then, that code could integrate with the test runner's reporter functionality. This would be a massive DX improvement. - Stabilize module mocking. If module: add clearCache for CJS and ESM node#61767 lands, I think it would make sense to leverage that first. I wanted to do something similar when I originally wrote the mocking code, but I anticipated there being FAR more pushback on breaking spec compliance then there has been in #61767. For ESM in particular, #61767 has the potential to greatly improve the DX of module mocking.
Some other nice to have changes would be:
- Migrate the test runner off of async_hooks. Now that
AsyncContextFrame, it may be possible to achieve the same functionality, and I believe the project would like to get rid of async_hooks. Currently, async_hooks are only used to map asynchronous activity back to the test that initiated it. I haven't investigated what it would take to make this change in much detail. - Give the spec reporter a new paint job. As long as you don't change the programmatic behavior of the reporter, we are free to change the actual output (text, colors, etc.) of the reporters without regard for semver. This has the potential to be a big DX improvement.
Activity
vassudanagunta commentedon Mar 7, 2026
Not a team member, but I'd like to take on nodejs/node#53103. My in-the-works
node-test-extralibrary, which uses a Proxy membrane to augmentnode:test, has this functionality implemented but it has problems with asynchronous tests executed by the--testrunner in subprocesses. I'd rather spend my time implementing it natively in node than figure out a solution in my proxy layer.I already have a custom spec reporter with a very pretty "paint job" that presents test-specific stdout in a nice way. Along with a PR for the above, I'd be able to offer a PR with a new spec reporter for your consideration.
JakobJingleheimer commentedon Mar 11, 2026
Thanks for this Colin!
This would be addressed by #13
This was raised at the last TC39 Harmony group meeting and it was well received and perceived to be doable (in particular, relaxing the requirement for dynamic imports to always return the same thing, which is what we'd need).
I think that is fair to say 😅
JakobJingleheimer commentedon Mar 11, 2026
No need to be a team member. By all means please do pick that up 🙂 I think it would be best started with a proposal for the change. We seem to have a good idea of the desired outcome, so perhaps TDD here would work well after the proposal is hashed out.
JakobJingleheimer commentedon Mar 11, 2026
Sorry, just noticed this bit too:
Could we maybe first get some broad-strokes of what it does? Assuming that's all good, then let's proceed with the PR 🙂
vassudanagunta commentedon Mar 11, 2026
In this particular case, I think it would be better if I started with a proposal. I'd like to get buy-in before I expend significant precious time.
vassudanagunta commentedon Mar 11, 2026
If the tests can be implemented with
node:test, the tests could come first in its own PR, all flaggedexpectFailure. That to me is perhaps the biggest value add of that new feature!