nx release publishes TS files in combination with default (integrated) generators

[lorenzodejong]

Current Behavior

Currently setting up a default (integrated) monorepo preset yields incorrect behavior when using nx release. By default, packages generated from @nx/js with the publishable option will generate a project.json which outputs changes to the /dist folder on the root of the monorepo. However the package.json in the src folder points to ./src/package.json.

Running the nx release command seems to be directly publishing the package from the source directory, causing the actual TypeScript source files to be published instead of the resolved build output in the dist folder.

It seems like the current implementation of nx release is only intended to be used in package-based repositories, where you specify your package.json to point to a dist folder present in the actual source folder.

The documentation also seems to be incorrect on this regard. If you look at the example of the published tarball you'll see that actual .ts files are getting published:

// ...
📦  @myorg/pkg-1@0.0.2
=== Tarball Contents ===

233B README.md
277B package.json
53B  src/index.ts
61B  src/lib/pkg-1.ts
=== Tarball Details ===
// ...

Expected Behavior

I expect nx release to be directly compatible with projects generated from the default provided Nx generators, without having to change the behavior of the package.json and project.json.

GitHub Repo

No response

Steps to Reproduce

1. Create a new repository with npx create-nx-workspace release-example-workspace, select to create an Integrated Monorepo when asked for it.
2. Generate a library with npx nx g @nx/js:lib mylib
3. Run npx nx release --first-release --dry-run
4. Run npx nx release publish --dry-run

Observed behavior: .ts files are present in the tarball.

Nx Report

Node   : 18.13.0
   OS     : darwin-arm64
   npm    : 8.19.3

   nx                 : 18.0.4
   @nx/js             : 18.0.4
   @nx/linter         : 18.0.4
   @nx/eslint         : 18.0.4
   @nx/workspace      : 18.0.4
   @nx/devkit         : 18.0.4
   @nx/eslint-plugin  : 18.0.4
   @nrwl/tao          : 18.0.4
   typescript         : 5.3.3

Failure Logs

No response

Package Manager Version

No response

Operating System

• macOS
• Linux
• Windows
• Other (Please specify)

Additional Information

No response

[cpierceworld]

I ran into the same issue with an Angular project with some "publishable" libraries. #21560

When I publish them (via either "nx release" or via "nx run my-library:nx-release-publish") the resulting npm package is the source code i.e. the "my-library/src/" folder. I expected it to publish the the compiled "dist/my-library"

[thdk]

@lorenzodejong are you trying to have a standalone nx workspace? Or a nx workspace that could eventually contain multiple packages? Your description and steps to reproduce are not completely clear for me.

However, as I'm an nx enthusiast, I saw your issue and tried to see if I could be of any help.
Therefor I have prepared a github repository containing a standalone typescript nx workspace:

https://github.com/thdk/nx-release-standalone-ts-library/tree/nrwl/nx/issues/21855

I have also documented my steps in the readme file in order to get it published using nx release command.

The final output looks like:

❯ npx nx release publish --dry-run

 >  NX   Running target nx-release-publish for project my-lib:

    - my-lib
   
   With additional flags:
     --dryRun=true

 ———————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————

> nx run my-lib:nx-release-publish


📦  @my-org/my-lib@0.0.1
=== Tarball Contents ===

1.1kB  LICENSE               
10.3kB README.md             
30B    dist/index.d.ts       
200B   dist/index.js         
116B   dist/index.js.map     
41B    dist/lib/my-lib.d.ts  
200B   dist/lib/my-lib.js    
172B   dist/lib/my-lib.js.map
10.6kB dist/README.md        
1.0kB  package.json          
=== Tarball Details ===
name:          @my-org/my-lib                          
version:       0.0.1                                   
filename:      my-org-my-lib-0.0.1.tgz                 
package size:  4.3 kB                                  
unpacked size: 23.8 kB                                 
shasum:        759b65eeb426b540ea9d93f946c0854f352dcd4b
integrity:     sha512-KLpammVULAbED[...]xZDugQ6nIdk4w==
total files:   10                                      
 
Would publish to https://registry.npmjs.org/ with tag "latest", but [dry-run] was set

 ———————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————

 >  NX   Successfully ran target nx-release-publish for project my-lib

There is one downside on my setup, it doesn't use the generated package.json file from the build target but used package.json file from the workspace root. I have found another way to make it work with the generate package.json as well but still unsure if that approach would be valid. As in, not documented, probably likely to break.

Update: below is the solution which sets the packageDir option for the nx-release-publish target to use the generated package.json for publishing:

Either configure per project in project.json

 "targets": {
    "nx-release-publish": {
      "options": {
        "packageRoot": "dist/{packageName}"
      }
    }
  },

or globally as targetDefaults in nx.json

 "targetDefaults": {
    "nx-release-publish": {
      "options": {
        "packageRoot": "dist/{packageName}"
      }
    }
  },

The result output for running nx release publish:

❯ npx nx release publish --dry-run

NX Running target nx-release-publish for project my-lib:

- my-lib

With additional flags:
--dryRun=true

————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————

nx run my-lib:nx-release-publish

📦 @my-org/my-lib@1.1.0
=== Tarball Contents ===

276B CHANGELOG.md
10.9kB README.md
30B index.d.ts
200B index.js
116B index.js.map
41B lib/my-lib.d.ts
200B lib/my-lib.js
172B lib/my-lib.js.map
976B package.json
=== Tarball Details ===
name: @my-org/my-lib
version: 1.1.0
filename: my-org-my-lib-1.1.0.tgz
package size: 3.6 kB
unpacked size: 12.9 kB
shasum: dceb978cfcb47d8f9f34d361e7dc6cec8dd9d80e
integrity: sha512-JlZ33VTeH+dD7[...]i+Gqn+593Rrgw==
total files: 9

Would publish to https://registry.npmjs.org/ with tag "latest", but [dry-run] was set

[lorenzodejong]

@thdk thanks for the extensive answer! I think my original message was indeed incorrect, i'm creating a monorepo with multiple packages. I think you have to choose for either a Package-based monorepo or a Integrated Monorepo to get this same behavior. I'll add that in my original reproduction steps.

The point you touch upon with the generated package.json might be a related issue indeed, where the package.json from the source is being used instead of the one from the build output.

[thdk]

Think I found the solution.

When running npx nx release publish you see eventually the following:

 >  NX   Successfully ran target nx-release-publish for project my-lib

Note the name of the target.

When I now look at the nx.json file from @nrwl/nx (this repo). I see the following:

    "targetDefaults": {
       "nx-release-publish": {
         "options": {
           "packageRoot": "build/packages/{projectName}"
         }
       },
    }

So assuming you have a folder packages containing your ts / js package source code, and each project is build output is placed in dist/packages/{projectName} you can adjust this configuration to match your own dist folder.

    "targetDefaults": {
       "nx-release-publish": {
         "options": {
           "packageRoot": "dist/packages/{projectName}"
         }
       },
    }

We can also tell Nx to bump the version in the generated package.json using the following config in nx.json:

  "release": {
     "git": {
        "commit": true,
        "tag": true
     },
    "projectsRelationship": "independent",
    "version": {
      "generatorOptions": {
        "specifierSource": "conventional-commits",
        "currentVersionResolver": "git-tag",
        "fallbackCurrentVersionResolver": "disk",
        "packageRoot": "dist/packages/{projectName}"
      }
    },
  }

Using conventional commits here to get the new version and git tags to find previously released versions since the version will not be bumped in the project's package.json file but only in the generated package.json. I believe other options are possible as well such as fetching existing version from the registry.

Example repo: https://github.com/thdk/nx-release-ts-packages

[acc-nicholas]

I'm running into the same root issue. After adding "packageRoot": "dist/libs/{projectName}" to my nx.json, that solves the problem of the correct compiled code packed and published to the registry.

But the next problem is that the version in the source code libs/foo/package.json gets updated via nx release version subcommand, but the dist/libs/foo/package.json does not get updated. So the publish fails because the dist version is stale and that version will already exist on the registry.

If i understand the comment above, you can workaround this issue by configuring release version to update the dist folder? But then the source folder is not matching still and that could lead to confusion as well? You'd just be relying on git tags to know what version your pkgs are on?

[lorenzodejong]

Yeah i also played around a bit more with this setup and i'm running into the same issue as @acc-nicholas is mentioning. The package.json in the dist gets updated, however isn't updated from a commit in the source folder. The command even fails because of this behavior, as it doesn't detect any git changes.

Also in the last comment of @thdk it's important to note that the configuration option is packageRoot instead of projectRoot:

  "release": {
     "git": {
        "commit": true,
        "tag": true
     },
    "projectsRelationship": "independent",
    "version": {
      "generatorOptions": {
        "specifierSource": "conventional-commits",
        "currentVersionResolver": "git-tag",
        "fallbackCurrentVersionResolver": "disk",
        "packageRoot": "dist/packages/{projectName}"
      }
    },
  }

[acc-nicholas]

i tried adding dependsOn to my nx.json, but that also didn't work.

"nx-release-publish": {
      "dependsOn": ["build"],
      "options": {
        "packageRoot": "dist/libs/{projectName}"
      }
    },

[thdk]

If i understand the comment above, you can workaround this issue by configuring release version to update the dist folder? But then the source folder is not matching still and that could lead to confusion as well? You'd just be relying on git tags to know what version your pkgs are on?

Sorry for the type in my previous comment (and linked repo). I have fixed that for future readers.

But you are right that, also as far as I know at the moment, with this setup git tags would become your source of truth. The version property of your package.json in libs/{projectName} would never be touched.

This now becomes the same issue as my raised issue here: #20936

[lorenzodejong]

@fahslaj could you perhaps provide some insight in the recommended approach here?

It feels like the nx release feature currently doesn't support a pretty standard publishing workflow: determining the next version of your package(s), compiling the code and publishing the compiled output with the corresponding package.json. Besides that the package.json in the package source code including the CHANGELOG.md should be updated.

Currently it's possible to set the packageRoot option, however that will point both publishing- and git commit logic to the dist folder. It feels like we're missing an option where we can:

• Specify the package source path for changelog and version creation (including creating a commit)
• Specify the package output path exclusively for publish

I would actually love to contribute here, it would be great if we can either support this workflow from the logic or write documentation on how to achieve this. Up until now i've had a pretty rough time discovering all the configuration options and the resulting output, so i feel like there's room for improvement.

[rainerhahnekamp]

I have the same problem and wonder how many cases we can publish the source code directly.

@JamesHenry, could you help us out here a little bit? Thanks!

[fahslaj]

@fahslaj could you perhaps provide some insight in the recommended approach here?

It feels like the nx release feature currently doesn't support a pretty standard publishing workflow: determining the next version of your package(s), compiling the code and publishing the compiled output with the corresponding package.json. Besides that the package.json in the package source code including the CHANGELOG.md should be updated.

Currently it's possible to set the packageRoot option, however that will point both publishing- and git commit logic to the dist folder. It feels like we're missing an option where we can:

• Specify the package source path for changelog and version creation (including creating a commit)
• Specify the package output path exclusively for publish

I would actually love to contribute here, it would be great if we can either support this workflow from the logic or write documentation on how to achieve this. Up until now i've had a pretty rough time discovering all the configuration options and the resulting output, so i feel like there's room for improvement.

Thank you for the clear explanation. I agree that this flow should be supported better. For now, as a workaround, you can create a release script that uses the programmatic api. It would call releaseVersion(...), then perform your build command, then releasePublish(...).

However, I do think this should be supported entirely from the CLI. I'll bring this up with the team and follow up when I have updates.

[lorenzodejong]

@fahslaj thanks for getting back to us!

That's indeed what i ended up doing, however it did take some extra effort to only build the specific packages that are actually about to be released.

I've created a gist with the current publish.ts script i've created: https://gist.github.com/lorenzodejong/acde50902b55ff2b72fffeeef2c35f9d.

For anyone interested on how to actually utilise it: you can use @swc/node to execute this script programatically:

$ node -r @swc-node/register tools/scripts/publish.ts

I've also included a specific flow in which a prerelease only creates a Git tag for version tracking purposes, whereas the latest release also includes a changelog and version bump in the package.json. This allows for the following CI/CD workflow:

• You can create a PR which incrementally publishes prereleases (1.0.0-PR123.0, 1.0.0-PR123.1, etc)
• After integrating the PR the actual version (1.0.0) gets released, creating a commit with the version bump and changelog and pushing it
• (Optionally) clean up the prerelease tags once integrated

For my team this is the ideal workflow, this allows you to test prerelease changes easily without being conflicted with version release commits on the PR branch.

Potential bug

One thing i've noticed however with this workflow is that an incorrect version gets released when creating the prerelease using the conventional-commits specifier source. Lets say the previous version is 0.1.0 and i create the following commit: feat!: this should become 1.0.0 it actually bumps it to 0.1.1-PR123.0 instead of 1.0.0-PR123.0.

@fahslaj i'm not sure if this bug has been reported already? Otherwise i'd be happy to create another more specific report for this with reproduction steps.

[fahslaj]

@lorenzodejong I am glad you were able to make it work with that script! As for the potential bug, if you could create a specific report with repro steps that would be greatly appreciated!

[rainerhahnekamp]

Hi @fahslaj, thanks for the prompt answer but I think that this should become top priority.

At the moment, nx release feels like a smartphone than can do everything except making phone calls... you know what I mean 😅

[JamesHenry]

@rainerhahnekamp let’s try and keep the discourse productive. typescript-eslint and angular-eslint are significant repos that are publishing their source files just fine with nx release, as are many others

[lorenzodejong]

It's actually awesome work you guys have put into this new feature, i can already see it's a very useful tool for our team.

Even though our workflow isn't fully supported yet from the default configuration, i do feel like it's close to being supported. That was also the reason for me to open this issue. If there's anything i can contribute to i'd be more than happy to take a shot at it.

I've opened an issue for the specific issue regarding the prerelease version resolver: #22150

[raketeFlo]

i tried adding dependsOn to my nx.json, but that also didn't work.

"nx-release-publish": {
      "dependsOn": ["build"],
      "options": {
        "packageRoot": "dist/libs/{projectName}"
      }
    },

I tried this as well and it actually solves the issue of an outdated dist folder. In my pipeline I run nx release version which updated the package.json and changelogs of my independent packages. I then go ahead and push the commit + tags to my remote. With the next step I run nx release publish which first builds all the affected packages and then publishes them. Here my nx.json:

{
  "targetDefaults": {
     "nx-release-publish": {
        "dependsOn": ["build"],
        "options": {
            "packageRoot": "{workspaceRoot}/dist/libs/{projectName}"
        }
     }
  },
  "release": {
    "projectsRelationship": "independent",
    "projects": ["libs/*", "!libs/workspace-extensions"],
    "releaseTagPattern": "{projectName}-{version}",
    "git": {
      "commitMessage": "chore({projectName}): release version {version} [skip-ci]"
    },
    "version": {
      "conventionalCommits": true
    },
    "changelog": {
      "projectChangelogs": {
        "createRelease": "github"
      }
    }
  }, 
}

[ryan-mcginty-alation]

@raketeFlo does the nx release publish command actually run the build or do you have to do that manually? I noticed you still have the dependsOn in place, but you also mentioned it didn't work. Just clarifying.

[raketeFlo]

@ryan-mcginty-alation nx release publish runs the build automatically because of the dependsOn. In my GitHub workflow I only run nx release version then I push the changes and tags and then I run nx release publish which builds my libs with the updated version and then publishes them.

The only thing I don't like about this approach is that the build could fail and then I already versioned and pushed the changes...

[jeffora]

@rainerhahnekamp let’s try and keep the discourse productive. typescript-eslint and angular-eslint are significant repos that are publishing their source files just fine with nx release, as are many others

@JamesHenry thanks for this comment, I've started looking at typescript-eslint as a reference in trying to implement nx release in my project. I notice that it only uses the programmatic API to achieve its release workflow, rather than the CLI.

Interestingly, it also uses tsc --build with project references rather than @nx/js:tsc with a root tsconfig with path mappings, and compiles dist folder side-by-side with source, rather than under a root dist/packages/{package-name}. These changes all seem to sidestep one of the issues discussed here, where the source-code package.json gets updated with release, but the dist package.json doesn't. However, these are all non-standard approaches if one has just scaffolded an nx repository by using the CLI defaults from the various generators.

Are these tweaks all necessary to get an nx release experience that requires minimal custom code? I.e. is it primarily suited for package-based monorepos rather than integrated monorepos that follow nx's more opinionated structure?

[zpydee]

I've been trying to solve this problem for the last couple of days (for a react vite library), and finally settled on the following:

1. I changed vite's outDir to './dist', placing the dist folder alongside the src folder in the library.
2. I added a .npmignore as follows:

src
.eslintrc.json
project.json
tsconfig.json
tsconfig.lib.json
vite.config.ts

3. I updated the package.json as follows:

{
  "name": "@spwntch/react-vite-sandbox",
  "version": "0.0.11",
  "main": "./dist/index.js",
  "module": "./dist/index.mjs",
  "types": "./dist/index.d.ts",
  "keywords": [
    "react",
    "vite",
    "sandbox"
  ],
  "author": "spwntch",
  "license": "MIT"
}

On publishing. this resulted in the following package, which I can consume:

📦  @spwntch/react-vite-sandbox@0.0.11
=== Tarball Contents ===

194B   README.md
42B    dist/index.d.ts
13.5kB dist/index.js
20.8kB dist/index.mjs
189B   dist/lib/react-vite-sandbox.d.ts
263B   package.json
=== Tarball Details ===
name:          @spwntch/react-vite-sandbox
version:       0.0.11
filename:      spwntch-react-vite-sandbox-0.0.11.tgz
package size:  9.9 kB
unpacked size: 34.9 kB
shasum:        532dbf531be7b2e3c95f80e7b1b88c329bc992c8
integrity:     sha512-xKdurjlp8+ej9[...]+7vLh8+gfKcLg==
total files:   6

Published to https://registry.npmjs.org/ with tag "latest"

Given the intent of project crystal, is this not a better approch than trying to mess with the nx release process?

[Jordan-Hall]

 "targetDefaults": {
   "nx-release-publish": {
      "dependsOn": [
        "build"
      ],
      "options": {
        "packageRoot": "dist/{projectRoot}"
      }
    }
 }

This repaired it for me. I want to say its just documentation issue rather than a actual bug

[joeflateau]

@fahslaj i see this is closed as completed, is this solution the recommended/correct approach? #21855 (comment)

[Jordan-Hall]

@fahslaj i see this is closed as completed, is this solution the recommended/correct approach? #21855 (comment)

#21855 (comment)

My comment above seems to me the solution. It's missing documentation maybe worth a PR to fix

[lorenzodejong]

This eventually also fixed the issue for me, however the only minor difference in my solution is that i targeted the packageRoot option to the source folder, e.g. {workspaceRoot}/packages/{projectName}.

The reason for this being that if you want to commit the updated CHANGELOG.md and version in the package.json using the automated commit feature from nx release, you want this to happen in the source folder of your code instead of the build output folder.

[acc-nicholas]

I still don't have a fix for this. Is my only option to spend time figuring out my own custom release script like @lorenzodejong has done? Not sure why this is closed.

The recommended fix of doing dependsOn: ["build"] doesn't even work as designed based on this comment

[fahslaj]

Reopening this, as I do think more documentation and clarification is needed. Depending on your workspace structure, @Jordan-Hall or @lorenzodejong 's solutions may work for you. There definitely needs to be more documentation on how Nx release handles the packageRoot and how to configure it according to your workspace setup.

In the meantime, this comment from James on another thread might shed some light on the matter: #20978 (comment)

Update: Additional documentation on the supported cases can be found on the docs site here.

[HosseinSalmanian]

@fahslaj I' still wondering does the problem solved by NX or we should rely on workarounds?
I've tried to use built-in functionality of NX for build and publishing libraries in mono repo for creating change log and bumping version automatically, but as others mentioned with referencing "packageRoot" to corresponding dist folder, version of published package does not bump.

[fahslaj]

@HosseinSalmanian If you need the version of the package within source control to be updated, but also need to publish a different packageRoot than the source, then you will need to use the nx release programmatic API along with your own scripting logic. This case is not currently supported by nx release CLI.

[arifshariati]

 "targetDefaults": {
   "nx-release-publish": {
      "dependsOn": [
        "build"
      ],
      "options": {
        "packageRoot": "dist/{projectRoot}"
      }
    }
 }

This repaired it for me. I want to say its just documentation issue rather than a actual bug

Normally, during CI we have a step for build, prior to release. In those cases, "dependsOn": ["build"] might not be required.

[Jordan-Hall]

 "targetDefaults": {
   "nx-release-publish": {
      "dependsOn": [
        "build"
      ],
      "options": {
        "packageRoot": "dist/{projectRoot}"
      }
    }
 }

This repaired it for me. I want to say its just documentation issue rather than a actual bug

Normally, during CI we have a step for build, prior to release. In those cases, "dependsOn": ["build"] might not be required.

While true this enforces it to run. Nx cache will just use the cache build

[ThePlenkov]

To my opinion it should work, but probably this code is the reason:

So basically there is a plugin which overwrites target defaults with dependsOn: ['^nx-release-publish']. Because of this it doesn't make sense what you put into target defaults - it's just ignored...

[ThePlenkov]

I commented this line in my local node_modules and everything worked well, release now works based on configured graph, not overwritten one: https://github.com/nrwl/nx/blame/c4c8b0150d827657632596656d92040108a338aa/packages/nx/src/utils/package-json.ts#L191

[ThePlenkov]

#22720 is a duplicate issue which is also impacted