Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to use platforms and toolchains to distinguish between Node and Browser targets? #2073

Closed
ajbouh opened this issue Jul 23, 2020 · 14 comments
Labels
Can Close? We will close this in 30 days if there is no further activity

Comments

@ajbouh
Copy link

ajbouh commented Jul 23, 2020

I am building a project that runs on both nodejs and in the browser. I need to use different implementations of a ts_library depending on the runtime environment.

This seems like a good fit for Bazel's platform concept, (in combination with ts_library's module_name argument), but I'm having trouble figuring out how to properly model this scenario.

Tools like webpack and rollup can be configured to look in the "browser" field of package.json to find an entry-point appropriate for the browser. I'd like to be able to accomplish the same in Bazel with rules_nodejs.

This project's README.md implies this should be possible but I can't find any guidance in the rest of the documentation on how to do it properly:

The nodejs rules integrate NodeJS development toolchain and runtime with Bazel.

This toolchain can be used to build applications that target a browser runtime ...

I've looked through the Bazel Platforms Cookbook but haven't seen anything there that would fit the bill.

@alexeagle
Copy link
Collaborator

We don't use Bazel's platform concept to model this. It produces distinct output trees for each platform assuming they are bitwise incompatible, where in JS we have mostly the same code with potentially many variations for different runtimes.

If you want to compile the same code to multiple target runtimes you just run the tool multiple times, being sure to have distinct output locations for each.

https://github.com/bazelbuild/rules_nodejs/blob/stable/examples/webapp/differential_loading.bzl is an example of creating both es5+systemjs and es2015+esm variants for old/modern browsers.

https://github.com/bazelbuild/rules_nodejs/blob/stable/packages/typescript/test/ts_project/outdir/BUILD.bazel#L4-L20 is an example of compiling with typescript to two different --module formats

https://github.com/bazelbuild/rules_nodejs/blob/065922b6963f1f9401c8a7638ef11c0709635886/docs/Rollup.md#rollup_bundle describes a technique for running rollup to get multiple output bundle formats

None of these is exactly the recipe for what you need but I hope it illustrates the pattern.

@ajbouh
Copy link
Author

ajbouh commented Jul 25, 2020 via email

@alexeagle
Copy link
Collaborator

I'm not sure which of these you mean:

  • use bazel to create and publish a library, the user of that library keys off the "browser" field in package.json and picks between one of two entry points in your library
  • the Bazel build needs to depend on a third-party library and read the "browser" field to choose an entry point that's used in one of the build steps

I'm guessing it's the first one. You can produce a package.json in your library, either just use the one you wrote, or if it needs to be dynamic you can generate one as a build step. A TypeScript compile rule just produces .js, you can have two of these to produce the different .js (maybe two entry point .ts files?) then combine that with a package.json in the final packaging step (pkg_npm maybe)

@ajbouh
Copy link
Author

ajbouh commented Aug 3, 2020

I'm actually trying to do a third thing, which is create packages that are either browser or node specific, but consume them exclusively via bazel dependencies.

When I try to do this I have two major issues:

  1. Setting up all the targets to properly depend only on the proper platform-specific version of a dependency (or the platform agnostic one)
  2. Errors from having the browser and the node targets both present in the dependency graph of a given target.

(1) is an issue because of the combinatorial explosion of dependencies and targets. Seems like being able to use select(...) would be much cleaner.

(2) is an issue because these packages have the same module_name for the browser and node versions, which creates a conflict deep in some nodejs/typescript rule.

The workflow I'm trying to iron out is essentially forking an existing project that depends on the browser field in package.json but keep all the target definitions and dependencies in "pure" bazel idioms.

@ajbouh
Copy link
Author

ajbouh commented Aug 7, 2020

Hi @alexeagle, any thoughts here?

@ashi009
Copy link
Contributor

ashi009 commented Aug 11, 2020

@ajbouh I believe we want the same thing. rules_go has something similar, setting goos, goarch on go_binary and those settings determine how go_library are built. This feature is documented on https://docs.bazel.build/versions/master/skylark/config.html. It's still lots to digest, but I think this is the right approach.

@ajbouh
Copy link
Author

ajbouh commented Aug 11, 2020

Oh! Yes the go rules are a good parallel.

I believe we should have build settings for browser and node targets. I think most of the value would be either in selecting srcs to include, or specifying rollup options to use. Perhaps even default typescript options?

Functionality along these lines would go a long way to making bazel an obviously better choice than the available alternatives in the greater JavaScript ecosystem.

@ajbouh
Copy link
Author

ajbouh commented Aug 13, 2020

I finally have a self-contained example of a project I'm trying to build for both browser and node environments.

To get things working I had to write a fairly large (and in my opinion, confusing) macro called iso_library

Under the hood this macro creates both platform-specific and platform-agnostic ts_library targets. Both the macro and the resulting build graph are much more complex than I'd like. I have a hunch that a select(...) based approach would be much better.

One major drawback of this macro approach is that it can't really be used between projects because of how many assumptions and implementation details I needed to hard-code to get things working.

I hope that this example code / project will help illuminate the core idea of this issue and perhaps give others some ideas about how they might deal with similar challenges.

@ashi009
Copy link
Contributor

ashi009 commented Aug 14, 2020

Based on the example you have, there is an alternative. Use the bundlers' static condition evaluation and dead code elimination feature to bundle correct code in the final output. Also, we put wrappers over those overlapping types eg. setTimeout, and we deal with those differences in those wrapper functions. (Very similar to the closure library's approach.)

For instance:

features.js

export const BROWSER_IOS = true;

lib.js

if (features.BROWSER_IOS) {
	console.log('ios');
} else {
	console.log('other');
}

lib.bundle.js

console.log('ios');

All we need is to generate that features file based on the build target, exporting all the constants to let bundler to do its work.

@ajbouh
Copy link
Author

ajbouh commented Aug 14, 2020

Thank you for the suggestion. Using bundlers as you suggest is certainly doable in a pure CommonJS/JavaScript world.

Unfortunately once Typescript gets involved... writing code that is type checked to work either in the browser or in node becomes challenging. You can no longer use type checking to prevent use of DOM types in node code. This is because Typescript won't limit global type definitions to just specific scopes.

Conditional imports also don't work with ES module imports.

My goal is to write a BUILD.bazel file for opentelemetry-js that doesn't require making large changes to the project layout.

(Edited for clarity)

@ashi009
Copy link
Contributor

ashi009 commented Sep 1, 2020

@ajbouh

After trying out https://github.com/bazelbuild/examples/tree/master/rules/starlark_configurations, here are some ideas:

  • Organize modules targeting different platforms in different file modules (also different ts_library), and make sure they implementations a common interface.
  • Export the common interface through the index.ts so that all the devtools are happy.
  • Use configuration to dynamically generate index.ts to export the implementation that matches the platform, and use the generated index.ts as the source of ts_library.

Also, --strictEnvironment is being considered, and global pollution should no longer be an issue if implemented.

@ajbouh
Copy link
Author

ajbouh commented Sep 1, 2020

Thanks for the great pointers- will definitely investigate!

@github-actions
Copy link

github-actions bot commented Nov 1, 2020

This issue has been automatically marked as stale because it has not had any activity for 60 days. It will be closed if no further activity occurs in two weeks. Collaborators can add a "cleanup" or "need: discussion" label to keep it open indefinitely. Thanks for your contributions to rules_nodejs!

@github-actions github-actions bot added the Can Close? We will close this in 30 days if there is no further activity label Nov 1, 2020
@github-actions
Copy link

This issue was automatically closed because it went two weeks without a reply since it was labeled "Can Close?"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Can Close? We will close this in 30 days if there is no further activity
Projects
None yet
Development

No branches or pull requests

4 participants