Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Created an NpxToolBase as an inheritable Subsystem for nodejs tools #17567

Merged
merged 3 commits into from
Nov 18, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/python/pants/backend/javascript/lint/prettier/rules.py
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ async def prettier_fmt(request: PrettierFmtRequest.Batch, prettier: Prettier) ->
result = await Get(
ProcessResult,
NpxProcess(
npm_package=prettier.default_version,
npm_package=prettier.version,
args=(
"--write",
*request.files,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,13 @@
import os
from typing import Iterable

from pants.backend.javascript.subsystems.npx_tool import NpxToolBase
from pants.core.util_rules.config_files import ConfigFilesRequest
from pants.option.option_types import ArgsListOption, SkipOption
from pants.option.subsystem import Subsystem
from pants.util.strutil import softwrap


class Prettier(Subsystem):
class Prettier(NpxToolBase):
options_scope = "prettier"
name = "Prettier"
help = softwrap(
Expand Down
6 changes: 5 additions & 1 deletion src/python/pants/backend/javascript/subsystems/nodejs.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
ExternalToolRequest,
TemplatedExternalTool,
)
from pants.core.util_rules.external_tool import rules as external_tool_rules
from pants.engine.fs import EMPTY_DIGEST, Digest, MergeDigests
from pants.engine.platform import Platform
from pants.engine.process import Process
Expand Down Expand Up @@ -109,4 +110,7 @@ async def setup_npx_process(


def rules() -> Iterable[Rule | UnionRule]:
return collect_rules()
return (
*collect_rules(),
*external_tool_rules(),
)
20 changes: 20 additions & 0 deletions src/python/pants/backend/javascript/subsystems/npx_tool.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Copyright 2022 Pants project contributors (see CONTRIBUTORS.md).
# Licensed under the Apache License, Version 2.0 (see LICENSE).

from __future__ import annotations

from typing import ClassVar

from pants.option.option_types import StrOption
from pants.option.subsystem import Subsystem


class NpxToolBase(Subsystem):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy to bikeshed on this name. NpxToolBase, NodeToolBase, NpmToolBase - I'm not sure which is the most precise, but extensible... Also may not matter, since it can always be changed later if need be.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd say NodeToolBase? We call the Python equivalent PythonToolBase, not PipToolBase or PyPIToolBase.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is somewhat complicated -- I suggest NpxToolBase, as npx is a tool launcher, and node is not (it's an arbitrary script launcher).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's an arbitrary script launcher

Feels like that would line up more with PythonToolBase in that case.

The only reason I was thinking NodeToolBase is that fundamentally, all of the tools need to be NodeJS tools. However, the same logic applies to npm and npx since it's the same ecosystem and suite of tools.

In Slack, @kaos mentioned

I’m not sure if there’ll ever be a risk for another node tool to not fit under that subsystem.
That's a point I just never thought about - and I don't know if it's the case or not.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Things launched by yarn come to mind as a good reason for being specific about the launcher.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Node is a runtime, so these are "tools that run on node" in the same way that PythonToolBases "run on the python interpreter" and JvmToolBases run on the JVM. So I think NodeToolBase is the right parallel?

Very roughly, Npx is to Node as Pip is to Python as Coursier is to JVM?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leaving NpxToolBase for now. Down the road when we think about potentially incorporating yarn, pnpm, or whatever the flavour of the month is in the future - we can revisit.

@benjyw npx isn't a package manager per se, but (more or less) an alias for "npm install" and then "npm exec" (https://docs.npmjs.com/cli/v7/commands/npx#npx-vs-npm-exec). So, npm is the pip.

This line of thinking has gotten me to the conclusion that NpxToolBase is better for now, specifically because it brings with it one piece of implicit info. The package MUST be executable with a bin folder/file specified in the package.json.

For example: npx dayjs errors out with npm ERR! could not determine executable to run.

So, NpmToolBase is incorrect, because the intention is an executable package. And for the same reason, I think NodeToolBase is incorrect, as you can put in Node packages which aren't executable, and then this ToolBase would be useless (without adding some extra flags, options, and shenanigans).

# Subclasses must set.
default_version: ClassVar[str]

version = StrOption(
advanced=True,
default=lambda cls: cls.default_version,
help="Version string for the tool in the form package@version (e.g. prettier@2.6.2)",
)
41 changes: 41 additions & 0 deletions src/python/pants/backend/javascript/subsystems/npx_tool_test.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Copyright 2022 Pants project contributors (see CONTRIBUTORS.md).
# Licensed under the Apache License, Version 2.0 (see LICENSE).

from __future__ import annotations

import pytest

from pants.backend.javascript.subsystems import nodejs
from pants.backend.javascript.subsystems.npx_tool import NpxToolBase
from pants.testutil.rule_runner import QueryRule, RuleRunner


class CowsayTool(NpxToolBase):
options_scope = "cowsay"
name = "Cowsay"
# Intentionally older version.
default_version = "cowsay@1.4.0"
help = "The Cowsay utility for printing cowsay messages"


@pytest.fixture
def rule_runner() -> RuleRunner:
return RuleRunner(
rules=[
*nodejs.rules(),
*CowsayTool.rules(), # type: ignore[call-arg]
QueryRule(CowsayTool, []),
],
)


def test_version_option_overrides_default(rule_runner: RuleRunner):
rule_runner.set_options(
[
"--backend-packages=['pants.backend.experimental.javascript']",
"--cowsay-version=cowsay@1.5.0",
]
)
tool = rule_runner.request(CowsayTool, [])
assert tool.default_version == "cowsay@1.4.0"
assert tool.version == "cowsay@1.5.0"
2 changes: 1 addition & 1 deletion src/python/pants/backend/python/typecheck/pyright/rules.py
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,7 @@ async def pyright_typecheck_partition(
process = await Get(
Process,
NpxProcess(
npm_package=pyright.default_version,
npm_package=pyright.version,
args=(
f"--venv-path={complete_pex_env.pex_root}", # Used with `venv` in config
*pyright.args, # User-added arguments
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,13 @@

from __future__ import annotations

from pants.backend.javascript.subsystems.npx_tool import NpxToolBase
from pants.backend.python.util_rules.interpreter_constraints import InterpreterConstraints
from pants.option.option_types import ArgsListOption, SkipOption, StrListOption
from pants.option.subsystem import Subsystem
from pants.util.strutil import softwrap


class Pyright(Subsystem):
class Pyright(NpxToolBase):
options_scope = "pyright"
name = "Pyright"
help = softwrap(
Expand Down