Repository navigation
Move nodejs/readable-stream CI from Travis to Jenkins #657
Description
Hey folks, do you think it might be possible to do this? What would it require from our part?
At this point we are evaluating for options, as we can't get a full CI run go green because of our current infrastructure.
Our current Travis setup is quite articulated: https://github.com/nodejs/readable-stream/blob/master/.travis.yml. And we would still need #655.
cc @nodejs/streams @addaleax
Activity
yoshuawuyts commentedon Mar 20, 2017
mcollina commentedon Mar 20, 2017
@yoshuawuyts There is also the fact that we are unable to trigger/manage the Travis integration, because of the lacking permissions in GitHub OAuth. At this point, we can't even restart a single job that failed.
I would like to know how much effort is needed for this, and then we can make our call.
yoshuawuyts commentedon Mar 20, 2017
gibfahn commentedon Mar 20, 2017
Related to: nodejs/readable-stream#265
There is now documentation for the process of adding new job types to the CI. @mhdawson has been handling it for other projects so far (currently citgm, node-report, and node-inspect).
Basic answer is that we have a reasonably good process in place now (covered in that document). We'd set up a team called
streams-admins(orreadable-stream-admins, whichever you'd like). Then we'd create jobs in ci.nodejs.org and give that team access.There are currently two types of jobs,
streams-continuous-integrationwould just pull a tar from nodejs.org (for when you're testing a change to readable-stream). Then there'd be astreams-smokerjob (for when you're testing that a change to Node doesn't break readable-stream). If this doesn't work we can always do something different, but not rebuilding node every time means a lot less stress on the CI (and much quicker CI runs).@phillipj is working on triggering CI runs with the GitHub bot through a comment by a collaborator for that repo in nodejs/github-bot#128, and also on having the job status reporting (i.e. what we have in node core).
The only thing I don't think we'll get that Travis has is automatically running CI on PRs (for obvious reasons). However we can run on all the platforms that Node supports, which is a big plus.
gibfahn commentedon Mar 20, 2017
@mcollina and to answer your question, it's not very much effort.
mcollina commentedon Mar 20, 2017
@gibfahn do you think it would be hard to have a localtunnel instance running somewhere as well? #655
Or maybe the servers have a public reacheable IP address, so there is no issue?
mcollina commentedon Mar 20, 2017
@gibfahn how would you setup those browsers runs that currently go to SauceLabs via localtunnel? As separate environments?
We are happy to pull Node.js, we just need to test this on all the Node.js versions we currently support, meaning 0.8, 0.10, 0.12, 1, 2, 3, 4, 5, 6, 7 (and soon 8). We do not care too much about multiple-platform at this point, as the source code is all JS.
readable-streamis a module on NPM (probably the most downloaded), so we need to test on what's released, so no building core stuff.This seems quite different from what you proposed.
gibfahn commentedon Mar 20, 2017
Yeah, that's what I meant by
streams-continuous-integration, pull a tarball from https://nodejs.org/dist/ and run your tests on it. See for example the equivalent for node-report. Different how?No idea, it's quite possible that your tests are doing something unusual that we'd need to make special arrangements for, could you give more details? cc/ @jbergstroem
gibfahn commentedon Mar 20, 2017
You'd also set up a pipeline that runs the same job with
0.8, 0.10, 0.12, 1, 2, 3, 4, 5, 6, 7as parameters, so you could just kick that off.mcollina commentedon Mar 20, 2017
I didn't understood :D.
Part of our current CI test suite opens up a server, exposed that to the world via localtunnel, then it has saucelabs connect there with various browsers, and that is published. This is not really working well for us on the localtunnel dependency.
gibfahn commentedon Mar 28, 2017
@mcollina Discussed this in the Build WG meeting (#660). Answer was that the localtunnel webserver shouldn't be a problem.
Whatever the better solution for this is, we should be able to accomodate on the build machines. Is there an issue somewhere to discuss alternatives for this? I'd like to understand in more detail.
mcollina commentedon Mar 29, 2017
Let's start with setting up
localtunnel, as it is causing massive issues even on Travis.yml #655. How can I help?gibfahn commentedon Mar 29, 2017
Let's first start with getting some Jenkins jobs set up and giving you (plural) access. I'll raise an issue to do that.
gibfahn commentedon May 5, 2017
Working on this with @mcollina and @piccoloaiutante in the Collab Summit, will update with progress.
Subtasks:
streams-adminsand give them accessmaclover7 commentedon Jun 1, 2018
@mcollina @gibfahn Just did a run of https://ci.nodejs.org/view/All/job/readable-stream-pipeline/ and it passed -- can this issue be closed?
mcollina commentedon Jun 1, 2018
Yes we can!