JS monorepos in prod 6: CI/CD, continuous integration and deployment with Travis CI

Implementing continuous integration CI and continuous deployment (CD) on a monorepo is quite complicated due to the diversity of multiple responsibilities between developers and the need to coordinate multiple packages. This article introduces the benefits of a monorepo integrated into a CI/CD pipeline. It tests and deploys such an application using Travis CI.

In it previous article discussing unit testing, we used Suede and Should.js to create unit tests. They were executed on our local machine. Continuous integration is the method of automating the execution of several tasks, including the execution of unit tests, whenever a commit is sent to a Git repository. The CI/CD pipeline that performs the tests provides feedback, helps integrate features, and automates the deployment of new releases.

This article is part of a series on JS monorepos best practices:

Run all tests at once

The lerna run command is handy for running the same Yarn or NPM command across all the packages it manages. The lerna run test perform test command from each package when it is registered.

A test script in the root package.json file records yarn test command:

{
  "scripts": {
    "postinstall": "husky install",
    "test": "lerna run test"  }
}

CI/CD enablement between GitHub and Travis CI

The project repository used to illustrate the article series can be found at GitHub. This article integrates the project with a CI/CD platform to automate the execution of tests and to publish new releases to The NPM Registry.

The CI/CD platform is Travis CI. Travis used to be free for all open source projects without any limitations. Following a recent acquisition and the goal of increasing profits (no judgment here), this offer has been limited. Since late 2020, Travis CI now provides a free plan for private and open source repos but it is limited to 10,000 credits. To learn more about Travis CI pricing, check out our their plans.

There are several CI/CD platforms you can choose from. We choose Travis because we have a long history of using it and for the quality and simplicity of the platform. The steps below are similar for alternative solutions. In a follow-up article, we'll achieve the same result using GitHub actions.

To use Travis CI, you don't need to create an account, just sign in with your GitHub account. Once you're logged in, go to your settings page where your repositories are synced from GitHub. If you can't find your archive, click the “Sync Account” button below your name. You will now be able to find your archive and activate it.

To instruct how to run the tests on the platform, Travis CI searches for one .travis.yml file in the root of the project. Our file is simple:

language: node_js
node_js:
  - '14'
cache:
  yarn: true
  directories:
    - "node_modules"
script:
  - yarn test

The yarn test on Travis CI is triggered when we push the change remotely on GitHub:

echo '!.travis.yml' >> .gitignore
git add .gitignore .travis.yml package.json
git commit -m "build: travis activation"
git push

When it's done, Travis informs the CI screen about the success of the execution.

Communicate about the CI status

You can insert a Travis CI badge in your readme file or anywhere else. It indicates the status of your current build:

<a href="https://travis-ci.org/#!/adaltas/remark-gatsby-plugins">
  <img src="https://api.travis-ci.org/adaltas/remark-gatsby-plugins.svg" alt="Build Status" />
a>

In Markdown, the badge looks like this:

[![Build Status](https://api.travis-ci.org/adaltas/remark-gatsby-plugins.svg)](https://travis-ci.org/#!/adaltas/remark-gatsby-plugins)

Run Travis CI locally with Docker

You can get a fully reproducible environment of your Travis Docker environment. A few weeks ago we share detailed instructions to recreate the same Travis Docker container on your local computer.

Continuous delivery

Tests are now performed on each building. Assuming a new version is in the current commit, we can issue lerna publish command from the CI/CD platform.

With Lerna we can use both lerna publish from-package and lerna publish from-git commands. I think the second one is better. It looks for the version tag in the current commit. If it exists, it extracts the version and publishes a new version to an NPM registry.

Let's consider our package ready to graduate and the prerelease cycle is there. For example, we want our title-to-frontmatter package to graduate from 1.0.0-alpha.3 to 1.0.0.

Travis CI must first be configured to deploy the package. Edit travis.yml by adding:

before_deploy:
  - echo "//registry.npmjs.org/:_authToken=\${NPM_TOKEN}" >> $HOME/.npmrc 2> /dev/null
deploy:
  provider: script
  script: "yarn run publish"
  on:
    node: "14"
    tags: true

The on property is to condition the execution of the deployment to the version of Node.js, in case we are testing against multiple versions, and the presence of tags in the current commit. The deployment only takes place when all conditions are met.

The provider and script properties indicate the use of the custom command yarn run publish to distribute our package. The package.json the file is updated to reflect the latest command:

{
  "scripts": {
    "postinstall": "husky install",    "publish": "lerna publish from-git --yes",
    "test": "lerna run test"
  }
}

Let's implement the changes:

git commit -a -m 'build: publish script'

Deploying a package in the NPM registry requires user credentials. Of course, we don't want to share our credentials in a public archive. Even in a private archive, you should never store any sensitive information such as passwords, secrets and certificates. Instead, we use environment variables configured in Travis CI. The good thing about environment variables when they are well declared, they don't need to appear in log files.

The NPM_TOKEN value is obtained from your NPM account. Go to the “Access Tokens” page and click “Generate new token”. Select the “Publish” type from the list. NPM states that the “Automation” token is “suitable for CI/CD workflows” but it just didn't work for me. I thought this is because 2FA is not yet enabled on my account.

Copy the value and paste it in the “Settings > Environment Variables” section of the Travis CI project:

The project has a new deploy instruction and NPM_TOKEN variable inside it is registered in Travis CI. We are now ready to deploy the new version. The lerna version command offers --conventional-graduate flag:

lerna version --conventional-commits --conventional-graduate
info cli using local version of lerna
lerna notice cli v3.22.1
lerna info versioning independent
lerna WARN conventional-graduate all packages
lerna info Graduating all prereleased packages
lerna info Looking for changed packages since gatsby-caddy-redirects-conf@0.1.0-alpha.3
lerna info ignoring diff in paths matching [ '**/test/**' ]
lerna info getChangelogConfig Successfully resolved preset "conventional-changelog-angular"

Changes:
 - gatsby-remark-title-to-frontmatter: 1.0.0-alpha.3 => 1.0.0
 - gatsby-caddy-redirects-conf: 0.1.0-alpha.3 => 0.1.0

? Are you sure you want to create these versions? Yes
lerna info execute Skipping releases
lerna info git Pushing tags...
lerna success version finished

The packs are now officially available to the community.

Cheating clap

• Add one test script in the root package.json file records yarn test command:

  {
    "scripts": {
      "postinstall": "husky install",
      "test": "lerna run test"
    }
  }

• Create an .travis.yml file in the root of the project to instruct how to run the tests on the platform:

  language: node_js
  node_js:
    - '14'
  cache:
    yarn: true
    directories:
      - "node_modules"
  script:
    - yarn test

• Commit the changes:

  echo '!.travis.yml' >> .gitignore
  git add .gitignore .travis.yml package.json
  git commit -m "build: travis activation"
  git push

• Add the tag to README.md:

  [![Build Status](https://api.travis-ci.org/adaltas/remark-gatsby-plugins.svg)](https://travis-ci.org/#!/adaltas/remark-gatsby-plugins)

• To deploy the package, edit .travis.yml by adding:

  before_deploy:
    - echo "//registry.npmjs.org/:_authToken=\${NPM_TOKEN}" >> $HOME/.npmrc 2> /dev/null
  deploy:
    provider: script
    script: "yarn run publish"
    on:
      node: "14"
      tags: true

• Add one more script package.json:

  {
    "scripts": {
      "postinstall": "husky install",
      "publish": "lerna publish from-git --yes",
      "test": "lerna run test"
    }
  }

• Commit the changes:

  git commit -a -m 'build: publish script'

• The NPM_TOKEN value obtained from the NPM “Access Tokens” page. Select “Publish” token or “Automation” token if 2FA is enabled. Copy the value and paste it into the “Settings > Environment Variables” section of the Travis CI project.

Conclusion

We went through how to implement a continuous integration workflow in your monorepo using Travis CI as well as how to do a continuous delivery process to deploy your package in the NPM registry.

Recently, Travis CI raises questions from the open source community on the future of its open source support. It follows their recent announcement of changes to Travis CI pricing. In response, they ensured that “resources are available to all developers and teams as well as improvements to the overall performance and stability of the platform”. But that's simply not what happens. For example, CI/CD pipelines for our open source projects started with delays of a few hours. As far as anyone can tell, they offer free minutes in fixed monthly allotments. There are only so many total minutes that they are willing to give out. These changes were initiated in 2019 when Travis CI was acquired by Idera, a private equity firm. However, it remains a good platform for non-open source projects.

In the next article, we'll cover CI/CD using GitHub Actions, a tool that makes it easy to automate software workflows.