Repository navigation
Windows CI is not working #3075
Description
At the time of this writing, https://ci.nodejs.org/job/node-test-binary-windows-native-suites/ has 36 consecutive failures. The last successful run was 34 hours ago.
I ran git-clean-windows and windows-update-reboot with force reboot in the hopes it would fix things, but it looks like it did not. (And I may have even made it worse? If so, sorry!) Prior to my trying it, https://ci.nodejs.org/view/All/job/windows-update-reboot/2147/ had been running for 4 days waiting for a host that was offline due to low disk space. I put that host back online hoping that a reboot would fix things.
Refs: https://openjs-foundation.slack.com/archives/C03BJP63CH0/p1668101438259619
Activity
[-]Windows CI has been down for 36 hours [/-][+]Windows CI is not working[/+]Trott commentedon Nov 11, 2022
It looks like @joaocgreis is on the case and things might be back to working. They ran git-clean-windows about an hour ago, although I'm sure they did other things too but that's the thing I noticed. Anyway, now test-rackspace-win2012r2_vs2019-x64-1 is 50+ minutes into a build whereas previously it had failed after 2.6 seconds on the same task.
Trott commentedon Nov 11, 2022
Alas node-test-binary-windows-native-suites and node-test-binary-windows-js-suites are still failing fast.
Trott commentedon Nov 11, 2022
@nodejs/build @nodejs/platform-windows I know today is a holiday in a lot of places, so I'm not sure if people are available to take a look at this, but just in case.... (And I think Joao may already be looking at it, but I'm not 100% certain. I inferred that from seeing a CI job they started.)
Trott commentedon Nov 12, 2022
I think the problem might be on ubuntu2204-x64-1 which those hosts use as a git server.
Caveat: I have no idea what I'm doing.
Trott commentedon Nov 12, 2022
It does seem like the permanently failing jobs are trying to check out things like
refs/heads/jenkins-node-test-commit-windows-fanned-0237cff7ff552f2db40f2cc16178eb6ecfc53670-bin-win-vs2019-x86when there isn't something withvs2019-x86at the end. There'svs2019-arm64andvs2019. I wonder if it's supposed to be using the latter, but not sure how/when this issue would have been introduced.Trott commentedon Nov 12, 2022
Ooh, there are
x86repos on there but the most recent one was created on November 9, which would seem to correspond with when this problem started. So I guess I need to go find the job that creates these repos and see if I can figure out why it stopped creating-x86branches?Trott commentedon Nov 12, 2022
@sxa It appears that you did a Jenkins configuration change right around this time, adding an entry for Windows ARM stuff. Is there any chance this is what caused this issue?
Trott commentedon Nov 12, 2022
This change in particular removes an
vs2019-x86reference. Maybe you meant to remove thevs2017-x86reference instead?Trott commentedon Nov 12, 2022
I added back win-vs2019-x86 and am running a fresh CI job. 🤞 https://ci.nodejs.org/job/node-test-commit-windows-fanned/51619/
Trott commentedon Nov 12, 2022
Putting it back seems to have done the trick. The tests are running on x86 where before, the machines would fail trying to use git (I guess to fetch the compiled executable or something similar to that).