Repository navigation
Build node with GN #21410
Description
Electron is starting to move its build system from GYP to GN to align with Chromium, which will make rolling Chromium substantially easier. However, we still have to shell out to GYP to build node.js, which is a lot of bridging code to maintain. I see there have been prior conversations about porting node's build system to GN from GYP, but they seem to have petered out without conclusion.
Would node.js be willing to accept patches to switch from GYP to GN?
NB, not suggesting that node does such a thing for 3rd-party native modules, just for node's own build.
Activity
bnoordhuis commentedon Jun 19, 2018
That's already possible with
./configure --build-v8-with-gn.edit: unless you mean porting all of the build system over. The answer is 'no' in that case.
nornagon commentedon Jun 19, 2018
Sorry, I'm not referring to building v8 with GN, I'm referring to building node.js itself with GN.
bnoordhuis commentedon Jun 19, 2018
Don't know if you saw my ninja edit but I think it's safe to say a switch to GN is not on the table.
nornagon commentedon Jun 19, 2018
Ah, no, I missed it (would be awesome if GH would refresh edits live). Could you expand on why that's the case? (Is there perhaps a document describing blockers/reasoning that you could refer me to?)
bnoordhuis commentedon Jun 19, 2018
See nodejs/TSC#464 which touches on that.
Speaking for myself, getting burned by a made-by-Google-for-Google build system once is enough; the next one should be one with broad community support.
mhdawson commentedon Jun 19, 2018
Based on earlier discussion Google said they don't support other projects using GN and recommended against it.
joyeecheung commentedon Jun 19, 2018
We are basically left with 3 choices after gyp:
./configure --build-v8-with-gnin our code base though. Future of addon tooling is bleak.See https://docs.google.com/document/d/101BP4BpZoP4tsMGo4j_MhoyLv169-2Oq_HeyWykCNGc/edit (OP of that TSC issue) for a breakdown
bnoordhuis commentedon Jun 19, 2018
Also apropos cmake, I'm reasonably confident I can teach node-gyp to convert a binding.gyp to CMakeLists.txt on the fly and effectively phase out GYP.
nornagon commentedon Jun 19, 2018
It's totally possible to use GN without depot_tools, see e.g. https://github.com/yue/build-gn. GN works fine on win/mac/linux, not sure about arm64 support but chromium itself builds against arm64: e.g.. @joyeecheung can you expand on what you mean by "platform support is limited"? Also not sure what you mean by "addon tooling".
joyeecheung commentedon Jun 19, 2018
Regarding OP's request:
I think patches for GN support can be accepted as long as they are not the default flow of building Node.js and are more in a situation similar to our Android builds (self-serve). If Electron needs GN to build Chromium then building Node.js with GN would be the most viable option for Electron, but it is not the case for Node.js itself and we have been explicitly advised against using it from the V8/Chromium/GN team.
joyeecheung commentedon Jun 19, 2018
Didn't know that, good to hear, thanks
https://docs.google.com/document/d/101BP4BpZoP4tsMGo4j_MhoyLv169-2Oq_HeyWykCNGc/edit touched on that a little bit (regarding FreeBSD and AIX). See supported platforms for the full list of platforms and tiers that we support.
I meant providing support for building 3rd-party native modules (as mentioned in the OP) with one of those tools and transitioning the ecosystem into it.
bnoordhuis commentedon Jun 19, 2018
Little to no freebsd/openbsd/etc. support. You're basically restricted to what Google is willing to support.
Supporting multiple build systems is already a hassle for libuv, and that's a fairly simple project to build.
I don't see it happening for Node.js. Without major ongoing maintenance it bitrots in no time.
joyeecheung commentedon Jun 19, 2018
I agree, but Idon't see any harm if we do not support GN, just allowing the build files to exist in the code base and get maintained by people who need them (e.g. Electron) though.
30 remaining items
mmarchini commentedon Oct 17, 2018
Does that mean we would need to use
gclient syncto build node, as well as have the entiredepot_toolsinstalled?Sanitizers come from GN or clang? If they come from clang we could probably use them with gyp. Also, I think I saw some PRs here to add support to build node with sanitizers.
To be clear, I'm not opposed to add GN to the project, I'm just trying to understand the implications.
refack commentedon Oct 17, 2018
Thank you very much @sam-github. I have voiced concern in the past that collab summit can turn into an exclusive decision making forum (IIUC the mandate is for high bandwidth discussion that is brought back to GitHub for consensus seeking). For the sake of inclusivity, I really appreciate you bringing the context back to the public!
👍 👍 👍
@hashseed (It is actually built 3 times: host&target for V8 and once more for node)
Anyway that's a bug, I'm sure we can fix anyway
IIUC no. If I'm wrong, and we do need depot_tools that is a problem.
hashseed commentedon Oct 17, 2018
Depot tools come with GN and ninja binaries, so yeah. Gclient sync is only necessary to fetch dependencies, which for Node we probably don't need, as the dependencies live in the same repository. Unless of course you want to ensure a particular version of clang to be fetched, like done for Chrome and V8.
hashseed commentedon Oct 17, 2018
I thought we only build separately for host and target for cross compiled builds, e.g. arm?
nornagon commentedon Oct 17, 2018
FYI, I've been working on a prototype of this here: https://github.com/electron/node/tree/gn-upstreaming
It's not ready to be a PR yet, but it does successfully build node (at least, on my machine 😉).
One tricky piece is that v8's GN files depend on Chromium's
build/infrastructure, which is fairly large and which I don't think would make sense to vendor into the node repo. I've solved that hackily in my current branch with a little setup script that fetches the repository, but there's probably a more sensible way to do it.joyeecheung commentedon Oct 17, 2018
I think by
around the table@sam-github was talking about this thread? There wasn't a physical table where we actually sat down and seeked consensus about GN v.s. GYP AFAIR (and most sessions were like brainstorming and demoing instead of consensus seeking). I didn't go to every session but I couldn't find any session specific to build in the agenda either.(At least all the tables that I could remember sharing with people in this thread had a lot of food on them and we weren't talking about code that much :S The sudden wave of replies probably stemmed from chats but AFAIK no decisions were attempted to be made there)
sam-github commentedon Oct 17, 2018
@joyeecheung there was a physical table, it was the LTS and Release group discussion. @addaleax was present and probably has a better memory than me who else was there.
joyeecheung commentedon Oct 17, 2018
@sam-github Thanks for the correction. Were there minutes taken? (Didn't find anything in nodejs/Release#358)
sam-github commentedon Oct 17, 2018
I don't recall. @MylesBorins kicked the meeting off, he may remember.
hashseed commentedon Jan 25, 2019
FWIW I hacked together a standalone Node build with GN and dependency management using gclient.
hashseed commentedon Feb 15, 2019
We are now running GN-built Node.js on V8' CI. That will soon replace the
--build-v8-with-gnoption.refack commentedon Apr 9, 2019
I have a slightly different approach in mind; teach GYP3 to parse GN syntax... AFAICT the delta in actual semantics is small.
This will not solve the V8 CI use case, but once we have GN parsing,
nodejs/node-v8&hashseed/gn-nodeshould give us good coverage.For the tooling integration use case, I'm planning on improving the "Compilation Database (a.k.a.
compile_commands.json) generator, as that seems to converge into a standard consumed by XCode, CLion, MSVS, and VSCode (as well as CLI tools, likeclang-tidy, and https://github.com/cquery-project/cquery).zuohuadong commentedon Jul 22, 2019
https://electronjs.org/blog/gn
GN +1
Node.js needs great changes!
Now, we are too slow.
vs Java / Dart / deno /Go
This seems too backward.
jasnell commentedon Jun 19, 2020
There's been no further activity here in nearly a year. Closing. Can reopen if absolutely necessary