Skip to content

Enable Arcade CI mode for Helix and code coverage - #7704

Merged
svick merged 11 commits into
dotnet:mainfrom
svick:fix/build-analysis-ci-mode
Oct 30, 2026
Merged

svick merged 11 commits into
dotnet:mainfrom
svick:fix/build-analysis-ci-mode

Conversation

@svick

@svick svick commented Sep 9, 2026 •

Copy link
Copy Markdown
Member

Fixes #7344.

Root cause

ML.NET invoked the Arcade Helix MSBuild wrappers without CI mode. When a Helix failure was already matched by a Known Build Error, the wrapper still returned exit code 1, causing Azure Pipelines to add a second generic Cmd.exe exited with code '1' or Bash exited with code '1' error. That generic error was not covered by the KBE, so Build Analysis remained red.

Changes

  • Invoke the Windows and Unix Helix wrappers with -ci / --ci. Arcade reports the underlying failure to Azure Pipelines but suppresses the redundant wrapper exit-code error.
  • Enable Arcade CI mode for the CodeCoverage build and test invocations.
  • Resolve NuGetPackageRoot and LightGBMVersion from evaluated MSBuild properties so CodeCoverage finds TorchSharp and LightGBM native assets in Arcade's repo-local CI package cache without duplicating its package-root logic.

Validation

  • Build 1589488 reproduced the original behavior: the deterministic failure was matched by temporary KBE [Temporary KBE] Build Analysis CI mode experiment for #7344 #7705, but the generic wrapper exit-code error remained unmatched.
  • Build 1590576 confirmed that Helix CI mode removes the generic Cmd.exe / Bash exited with code '1' error.
  • Build 1595561 passed after the temporary experiment test was removed.
  • CodeCoverage build 1597259 passed with CI mode and dynamically resolved TorchSharp and LightGBM native paths.
  • Clean MachineLearning-CI rerun 1597536 passed against the final merge commit. The GitHub Build Analysis check remains associated with build 1597258, whose only failed leg hit an unrelated transient connection failure while downloading .NET runtime 8.0.16.

Remaining caveat

The legacy Helix Azure Pipelines reporter can still emit a separate structured Test <name> has failed error that is not necessarily matched by the KBE. This PR deliberately does not disable that reporter because doing so would change which system gates test failures. Adopting the standalone Helix Job Monitor will disable the legacy reporter and remove that remaining duplicate error path.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove the temporary Build Analysis experiment test now that the CI behavior has been captured.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
svick and others added 2 commits September 11, 2026 17:46
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@codecov

codecov Bot commented Sep 15, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 70.81%. Comparing base (44ede55) to head (8e81265).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #7704      +/-   ##
==========================================
- Coverage   70.81%   70.81%   -0.01%     
==========================================
  Files        1421     1421              
  Lines      280034   280034              
  Branches    27621    27621              
==========================================
- Hits       198304   198301       -3     
+ Misses      73568    73563       -5     
- Partials     8162     8170       +8     
Flag Coverage Δ
Debug 70.81% <ø> (-0.01%) ⬇️
production 64.25% <ø> (+<0.01%) ⬆️
test 90.75% <ø> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 12 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@svick svick changed the title Test Build Analysis handling with Arcade CI mode Enable Arcade CI mode for Helix and code coverage Sep 16, 2026
@svick
svick marked this pull request as ready for review September 16, 2026 11:19
Copilot AI lite review requested due to automatic review settings September 16, 2026 11:19

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Fix the Unix-incompatible script syntax and centralize the Torch native package versions.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Enables Arcade CI mode for Helix and code-coverage workflows while configuring native dependency paths.

Changes:

  • Adds platform-specific CI flags to build, test, and Helix invocations.
  • Resolves package-root and LightGBM paths dynamically for coverage.
  • Configures native TorchSharp dependencies for coverage tests.
File summaries
File Summary Findings
build/ci/job-template.yml Updates shared CI, Helix, and coverage commands. Critical: Windows cmd.exe syntax is used in Unix Bash steps. Moderate: Torch native package versions remain hard-coded instead of using centralized properties.
Review details
  • Files reviewed: 1/1 changed files
  • Comments generated: 2
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread build/ci/job-template.yml Resolved
Comment thread build/ci/job-template.yml Resolved
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: c0435a7a-e09d-4246-a749-c89d64d47f4f
Comment thread build/ci/job-template.yml Resolved
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: c0435a7a-e09d-4246-a749-c89d64d47f4f
@svick
svick requested review from rosebyte and a balanced review from Copilot October 14, 2026 13:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The changes match the stated CI failure mode and validated package-cache requirements without unresolved issues.

0 open findings

2 resolved since last review

🧠 Review effort: Balanced

@matouskozak

Copy link
Copy Markdown
Member

Should we enable the -ci flag for the official CI also? (build/vsts-ci.yml). I'm not sure it uses the job-template...

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: c0435a7a-e09d-4246-a749-c89d64d47f4f
@svick

svick commented Oct 14, 2026

Copy link
Copy Markdown
Member Author

@matouskozak Good point. I think they should, so I enabled them.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No blocking issues remain, and the reported CI validation supports the changes in active pipeline configurations.

0 open findings

🧠 Review effort: Balanced

Comment thread build/ci/job-template.yml
$packageRoot = $properties.NuGetPackageRoot
$env:PATH += ";${packageRoot}libtorch-cpu-win-x64\$($properties.LibTorchVersion)\runtimes\win-x64\native"
$env:PATH += ";${packageRoot}torchsharp\$($properties.TorchSharpVersion)\runtimes\win-x64\native"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these the only native libraries we should add? I think TensorFlow also has some dependency.

@svick svick Oct 20, 2026 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TensorFlow does have a native dependency, but it doesn't need to be added here, the dependency can be correctly resolved already.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@svick svick Oct 29, 2026 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, TensorFlow should be added here, so I did that.

There are other TensorFlow (and more) tests that are still skipped. I will open a separate PR once this one is merged.

Comment thread build/ci/job-template.yml Resolved

@rosebyte rosebyte left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, please take a look on Matous' comments before merging.

@svick
svick requested review from matouskozak and rosebyte and a balanced review from Copilot October 20, 2026 13:49

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No blocking issues remain, and the reported CI validation supports the intended behavior.

No open issues 🎉

Review effort: Balanced

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 0c982995-379f-4379-bcb7-dba5d1102bb4
@svick
svick marked this pull request as draft October 29, 2026 13:24
@svick
svick marked this pull request as ready for review October 29, 2026 14:37
@svick
svick merged commit d6f193e into dotnet:main Oct 30, 2026
27 checks passed
@svick
svick deleted the fix/build-analysis-ci-mode branch October 30, 2026 15:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Run helix tests still fails the build when all issues are known

4 participants