Updating Node with NVM on the Server

This wiki page relates to the code put forth in PR's 511, 505, and 497:

Our server had been set to use Node version 12.22.12, and was the same for all apps on the server. The long-term support (LTS) version of Node is currently 20, with version 22 released and intended to become the next LTS version. Our older version 12.22.12 did not support several dependencies needed to upgrade the app.

We talked about this problem and decided to change our server from serving one specific Node version to using NVM with a default, thus allowing individual apps to use a .nvmrc file to pick a specific Node version to use.

There are some code changes needed to be able to take advantage of our new setup:

The `.nvmrc` File

Using NVM and the .nvmrc file

If you wanted to change what version of Node you were using on your local machine to version 20.14.0, you would run nvm use 20.14.0. With the addition of the .nvmrc file, you now run nvm use. NVM will look for a .nvmrc file in the directory you're in and will apply the version written in the file.

The new .nvmrc file

A .nvmrc file is very simple - you simply put in what version of node you want to use, a blank line after, and nothing else. Our app uses Node version 20.14.0, so this is our .nvmrc file:

20.14.0

The `.circleci` File

Previously, we had not been attempting to keep our CircleCI node version in line with our deployment versions. When we moved to Ruby version 3.3.1, in PR 505, the Node version we had been using in CircleCI was unable to handle installing Ruby 3.3.1. To handle this, we first had CircleCI download NVM and then Node version 12.22.12, which indeed can handle Ruby 3.3.1. In PR's 511 and 497 we set CircleCI to use NVM and the Node version specified in the .nvmrc file.

Key Changes

Install the NVM and the correct node version:

      - run:
          name: Install NVM and Node.js
          command: |
            curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash
            echo 'export NVM_DIR="$HOME/.nvm"' >> $BASH_ENV
            echo '[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" --install' >> $BASH_ENV
            echo '[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"' >> $BASH_ENV
            source $BASH_ENV
            nvm install $(cat .nvmrc)
            nvm use $(cat .nvmrc)

Every time you have a new run command, CircleCI treats it as if you had opened a new console. Fortunately, you have saved your new node installations and the version you need is available to be called if you add the following to the top of the inside of any run command. You need to add source $BASH_ENV to make nvm available, and then nvm use to use the version of node in your .nvmrc file.

For example, our run command for installing our Gemfile dependencies looks like this:

      - run:
          name: Install Gemfile Dependencies
          no_output_timeout: 15m
          shell: /bin/bash -eo pipefail
          command: |
            source $BASH_ENV
            nvm use
            echo "Installing Ruby gems..."
            bundle install --jobs=4 --retry=3

Our Complete `.circleci/config.yml` File

# Ruby CircleCI 2.0 configuration file
#
# Check https://circleci.com/docs/2.0/language-ruby/ for more details
#
version: 2.1
executors:
  docker-publisher:
    environment:
      IMAGE_NAME: treatment-database-app
    docker:
      - image: docker:20.10.14-git

orbs:
  ruby: circleci/ruby@2.1.1
  browser-tools: circleci/browser-tools@1.4.7
  coveralls: coveralls/coveralls@1.0.6

jobs:
  build:
    docker:
      # specify the version you desire here
      - image: cimg/ruby:3.3.3

      # Specify service dependencies here if necessary
      # CircleCI maintains a library of pre-built images
      # documented at https://circleci.com/docs/2.0/circleci-images/
      # - image: circleci/postgres:9.4

    environment:
      BUNDLE_PATH: vendor/bundle
      BUNDLE_JOBS: 4
      BUNDLE_RETRY: 3
      RAILS_ENV: test
      RACK_ENV: test
      SPEC_OPTS: --profile 10 --format RspecJunitFormatter --out /tmp/test-results/rspec.xml --format progress
      WORKING_PATH: /tmp
      UPLOAD_PATH: /tmp
      CACHE_PATH: /tmp/cache
      COVERALLS_PARALLEL: true


    working_directory: ~/treatment_database

    steps:
      - checkout
      - browser-tools/install-browser-tools

      - restore_cache:
          keys:
            - v1-dependencies-{{ checksum "Gemfile.lock" }}
            # fallback to using the latest cache if no exact match is found
            - v1-dependencies-

      - run:
          name: Configure Bundler
          command: |
            echo 'export BUNDLER_VERSION=$(cat Gemfile.lock | tail -1 | tr -d " ")' >> $BASH_ENV
            source $BASH_ENV
            gem install bundler -v "$(grep -A 1 "BUNDLED WITH" Gemfile.lock | tail -n 1)"

      - run:
          name: Install NVM and Node.js
          command: |
            curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash
            echo 'export NVM_DIR="$HOME/.nvm"' >> $BASH_ENV
            echo '[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" --install' >> $BASH_ENV
            echo '[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"' >> $BASH_ENV
            source $BASH_ENV
            nvm install $(cat .nvmrc)
            nvm use $(cat .nvmrc)
            echo "Node Version Manager (NVM) installed successfully."
            echo "NVM version: $(nvm --version)"
            echo "Node.js version: $(node --version)"

      - run:
          name: Install Yarn
          command: |
            source $BASH_ENV
            nvm use
            npm install -g yarn
            echo "Yarn installed successfully."
            echo "Yarn version: $(yarn --version)"

      - run:
          name: Install Gemfile Dependencies
          no_output_timeout: 15m
          shell: /bin/bash -eo pipefail
          command: |
            source $BASH_ENV
            nvm use
            echo "Installing Ruby gems..."
            bundle install --jobs=4 --retry=3

            echo "Installing required system packages..."
            sudo apt-get update
            sudo apt-get install -y xvfb libfontconfig wkhtmltopdf

            echo "Installation steps completed successfully."

      - run:
          name: Install Yarn Dependencies
          command: |
            echo "Installing Yarn dependencies..."
            source $BASH_ENV
            nvm use
            yarn install

      - run:
          name: Run Yarn Build
          command: |
            echo "Running Yarn build..."
            source $BASH_ENV
            nvm use
            yarn build

      - save_cache:
          paths:
            - ./vendor/bundle
          key: v1-dependencies-{{ checksum "Gemfile.lock" }}

      # Database setup
      - run: bundle exec rake db:create
      - run: bundle exec rake db:schema:load

      - run:
          name: Rubocop
          command: |
            gem install rubocop
            bundle exec rubocop --require rubocop-rails

      # Brakeman
      - run:
          name: Run Brakeman
          command: bundle exec brakeman -q -w 2

      # Bundler-audit
      - run:
          name: Install Bundler-audit
          command: gem install bundler-audit
      - run:
          name: Run Bundle-audit
          command: bundle exec bundle audit check --update

      # run tests!
      - run:
          name: Run rspec in parallel
          command: |
            mkdir /tmp/test-results
            bundle exec rspec $(circleci tests glob "spec/**/*_spec.rb" | circleci tests split --split-by=timings)
            #bundle exec rspec --out /tmp/test-results/rspec.xml $(circleci tests glob "spec/**/*_spec.rb" | circleci tests split --split-by=timings)

      - coveralls/upload:
          parallel_finished: true
          path_to_lcov: /home/circleci/treatment_database/coverage/lcov/treatment_database.lcov

workflows:
  version: 2
  ci:
    jobs:
      - build

Using Node in the Capistrano Deploy

When you run cap qa deploy or cap production deploy, Capistrano will automatically use the default version of Node that we have on the server (currently 12.22.12), and it actually will use that version instead of the node version in your .nvmrc file for installing your dependencies. If your dependencies can deploy on version 12.22.12, you can stop reading here - you're done.

Capistrano does its deployment by running a series of "rake tasks", similar to each "run" task we had for CircleCI. And, as with CircleCI, we need to find a way to ensure that each task has access to nvm and the correct version of node. Unfortunately, it is impractical to find and overwrite each and every rake task to ensure correct deployment. Tasks can be buried within other tasks, and things can get very confusing very quickly.

To handle this issue, we need to get access to NVM and our preferred node version, and then load nvm and our node version within each rake task. Here is the solution we are using in Treatment Database:

Install NVM and the node version we wish to use.

We created a custom rake task called nvm:load. It loads NVM (already installed on the server) and installs the node version on the server if it's not already installed. Here is the task, located at /lib/capistrano/tasks/nvm.rake:

# frozen_string_literal: true

namespace :nvm do
  task :load do
    on roles(:all) do
      within release_path do
        execute :echo, 'Sourcing NVM, installing Node version, and setting Node version'
        execute "source ~/.nvm/nvm.sh && nvm install $(cat #{release_path}/.nvmrc) && nvm use $(cat #{release_path}/.nvmrc)"
      end
    end
  end
end

This task loads nvm from the server and installs the version of node from our .nvmrc file if it's not already present. This is necessary because we cannot guarantee that the server will already have our version of node available. Unfortunately, it does NOT make Capistrano use that version of node in all subsequent tasks. For that, we have our next custom task...

Load NVM and our node version within each task

To make sure that we are using the version of node that we just installed, we need to find a way to tell each and every "execute" command in our tasks to find and use our node version. To do this, we need to create a task that will modify how commands are actually called, prepending instructions to find nvm and then use our node version. That's what this task does. It's located at /lib/capistrano/tasks/nvm_integration.rake

# frozen_string_literal: true

namespace :nvm do
  task :setup do
    on roles(:web) do
      SSHKit.config.command_map.prefix[:rake] ||= []

      begin
        execute :echo, 'Checking .nvmrc presence...'
        if test("[ -f #{release_path}/.nvmrc ]")
          execute :echo, 'Sourcing NVM and setting Node version...'
          SSHKit.config.command_map.prefix[:rake].unshift("source ~/.nvm/nvm.sh && nvm use $(cat #{release_path}/.nvmrc) &&")
        else
          error "No .nvmrc file found in #{release_path}"
          exit 1
        end
      rescue SSHKit::Command::Failed => e
        error "NVM setup failed: #{e.message}"
        raise
      end
    end
  end
end

This task modifies the "command" that SSHKit uses to execute its instructions in each rake task. It adds source ~/.nvm/nvm.sh && nvm use $(cat #{release_path}/.nvmrc) && to the front of each command, which loads nvm and runs nvm use with your version of .nvmrc. This change will remain in place throughout the duration of the Capistrano deploy, but does not persist to the next deployment.

Use hooks to execute the custom tasks

We have created two custom tasks above, but we never actually tell them to execute. That's as useful as writing a function but never calling it. In order to "call" these tasks, we use "hooks". In the file at /config/deploy.rb, we need to add the following lines:

after 'git:create_release', 'nvm:load'
after 'nvm:load', 'nvm:setup'

That will install and load our node versions at the beginning of the deploy process, ensuring that we are using the correct version of node throughout our deploy.

Loading the custom tasks

Finally, we need to make sure that our lovely rake tasks are actually loaded. This is handled in a top-level file named Capfile. In Capfile, we need to add the following lines if not already present:

# Include tasks from the `lib/capistrano/tasks` directory
Dir.glob('lib/capistrano/tasks/*.rake').each { |r| import r }

This makes each rake task (saved as file-name.rake) within the lib/capistrano/tasks directory available. If you do that, Rails can handle the rest of the work of finding each task and using it when called.

If your deploy is failing

There is a chance you will need to add the following to your Gemfile. At this point I'm not certain which are actually needed for the changes above, since the nvm integration needed to be corrected several times...

group :development do
  gem 'capistrano-bundler', require: false # Capistrano integration for Bundler
  gem 'capistrano-rails', require: false # Integrates Rails with Capistrano
  gem 'capistrano-rvm', require: false # RVM integration for Capistrano
  gem 'capistrano-spec' # RSpec matchers for Capistrano

Using `yarn build`

Here's what we needed to do to get `yarn build` to execute properly:

#  frozen_string_literal: true

puts 'Loading Yarn tasks...'
namespace :yarn do
  desc 'Build yarn packages'
  task :build do
    on roles(:all) do
      within release_path do
        execute :echo, 'Sourcing NVM and running yarn build'
        execute "source ~/.nvm/nvm.sh && nvm use $(cat #{release_path}/.nvmrc) && cd #{release_path} && RAILS_ENV=production yarn build"
      end
    end
  end
end

It appears that we have some unnecessary duplication of the sourcing of our correct node version, but when source ~/.nvm/nvm.sh && nvm use $(cat #{release_path}/.nvmrc) && is removed, the deploy fails. Since it works as-is, the file is staying like this for now.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updating Node with NVM on the Server

The `.nvmrc` File

Using NVM and the .nvmrc file

The new .nvmrc file

The `.circleci` File

Key Changes

Using Node in the Capistrano Deploy

Install NVM and the node version we wish to use.

Load NVM and our node version within each task

Use hooks to execute the custom tasks

Loading the custom tasks

Clone this wiki locally

Updating Node with NVM on the Server

The .nvmrc File

Using NVM and the .nvmrc file

The new .nvmrc file

The .circleci File

Key Changes

Using Node in the Capistrano Deploy

Install NVM and the node version we wish to use.

Load NVM and our node version within each task

Use hooks to execute the custom tasks

Loading the custom tasks

Clone this wiki locally

The `.nvmrc` File

The `.circleci` File