DEV Community

Thiago Marinho
Thiago Marinho

Posted on

How to Deploy a Monorepo with TurboRepo on Heroku

#github #heroku #monorepo

Feel free to read it in my personal blog tgmarinho.com

Intro

I replaced four (sdk, smart-contract, indexer-api, frontend app) GitHub projects with only one using Monorepo / TurboRepo.

I did a post about it. Check it out

The structure of my monorepo:



~/Developer/blog/monorepo (main*) » tree -L 3 --gitignore
.
├── README.md
├── apps
│   ├── frontend
│   │   ├── Procfile
│   │   ├── README.md
│   │   ├── __mocks__
│   │   ├── __tests__
│   │   ├── next-env.d.ts
│   │   ├── next.config.js
│   │   ├── package.json
│   │   ├── public
│   │   ├── src
│   ├── contract
│   │   ├── README.md
│   │   ├── contracts
│   │   ├── hardhat.config.ts
│   │   ├── package.json
│   │   ├── scripts
│   └── backend
│       ├── README.md
│       ├── Procfile
│       ├── build
│       ├── package.json
│       ├── src
│       ├── tsup.config.ts
├── package.json
├── packages
│   ├── contract-types
│   │   ├── README.md
│   │   ├── package.json
│   │   ├── src
│   │   └── tsconfig.json
│   ├── sdk
│   │   ├── README.md
│   │   ├── jest.config.js
│   │   ├── jest.setup.js
│   │   ├── package.json
│   │   ├── src
│   │   └── tsup.config.ts
│   ├── eslint-config-custom
│   │   ├── index.js
│   │   └── package.json
│   └── tsconfig
│       ├── README.md
│       ├── base.json
│       ├── nextjs.json
│       ├── package.json
│       └── react-library.json
├── turbo.json
└── yarn.lock

// Omitting some files and packages unnecessary to this blog post
Enter fullscreen mode Exit fullscreen mode

A little bit of the context

The hard part was hosting, and I'll tell you how to do it on Heroku.

But before, I wish to give you some context on how this project should do in the build process:

The smart-contract should do the build process to generate all types using typechain lib, because instead of using the ABI, I want to use the types (typescript for the win); this build process generates the folder types inside of the
smart-contract project; and my script copies this types folder and pastes it to new packages called contract-types (that should be an npm package of types).

The sdk should do the build process and use the contract-types, then the frontend app should build and use the sdk that uses the contract-types.

The indexer-api (backend) should build using the contract-types.

Order of the build:

  1. smart-contract
  2. the types of contract-types should exist
  3. sdk
  4. parallel frontend and backend

TurboRepo does it in a fast and intelligent way without much effort.

There are others things I'm omitting because they are not too important, but we have other packages.

With this in mind, let's see how to set up the Heroku to work on monorepo:

Deployment - Create the Apps

Create two apps on heroku:

  1. frontend - heroku create -a frontend
  2. backend - heroku create -a backend

Add Buildpacks

In both apps, you can connect Heroku apps to the Github. This way, you will save time with CI/CD after committing to the main branch.
Also, in both ones, you need to follow the same steps:

Add (GUI: settings -> buildpacks -> Add Buildpack) the buildpacks in this order:

  1. https://github.com/heroku/heroku-buildpack-multi-procfile
  2. heroku/nodejs

Or Heroku CLI: 



heroku buildpacks:add -a frontend heroku-community/multi-procfile
heroku buildpacks:add -a frontend heroku/nodejs

heroku buildpacks:add -a backend heroku-community/multi-procfile
heroku buildpacks:add -a backend heroku/nodejs
Enter fullscreen mode Exit fullscreen mode

Create the Procfile

Procfile is a file that receives the commands to run when starting an application; if you have a basic node.js Heroku app, you don't need this once the package.json has the start script instruction.

But in our case, we'll need this for frontend and backend packages:

Frontend:



echo "web: cd apps/frontend && yarn start" > Procfile
Enter fullscreen mode Exit fullscreen mode

Backend:



echo "web: cd apps/backend && yarn start" > Procfile
Enter fullscreen mode Exit fullscreen mode

The command above creates the Procfile file with the content: web: cd apps/backend && yarn start

Setup the new env PROCFILE for file path to Procfile:

Frontend App:



heroku config:set -a frontend PROCFILE=apps/frontend/Procfile
Enter fullscreen mode Exit fullscreen mode

Backend App:



heroku config:set -a backend PROCFILE=apps/backend/Procfile
Enter fullscreen mode Exit fullscreen mode

Setup the root package.json on monorepo

Heroku now knows where to find our Procfiles; however, because we have two separate applications stored within the frontend (client) and backend (server) directories, each has its dependencies.

Heroku typically tries to install dependencies as specified in the package.json at the project's root and will try to run the build script set here. To ensure we install the correct dependencies and run the proper build scripts for our application, we need to specify a heroku-postbuild script at the root of our project.

The secret ingredient of the recipe: In the package.json file in the project root, add the following scripts:



"build:frontend": "turbo run build --filter=frontend",
"build:backend": "turbo run build --filter=backend",
"heroku-postbuild": "if [ $CLIENT_ENV ]; then yarn run prod-frontend; elif [ $SERVER_ENV ]; then yarn run prod-backend; else echo no environment detected, please set CLIENT_ENV or SERVER_ENV; fi",
"prod-frontend": "yarn run build:frontend",
"prod-backend": "yarn run build:backend"
Enter fullscreen mode Exit fullscreen mode

We’ve added three scripts: heroku-postbuild, prod-frontend, and prod-backend.

Heroku will automatically run the heroku-postbuild script for us upon deployment.

Our heroku-postbuild script looks for environment variables $CLIENT_ENV or $SERVER_ENV to determine which script to run prod-frontend or prod-backend.

Setting environment variables on Heroku

Now add new CLIENT_ENV and SERVER_ENV on heroku apps:

Frontend App:



heroku config:set -a frontend CLIENT_ENV=true
Enter fullscreen mode Exit fullscreen mode

Backend App:



heroku config:set -a backend SERVER_ENV=true
Enter fullscreen mode Exit fullscreen mode

Now our heroku-postbuild script will be able to run the correct install scripts for each of our applications on deployment.

See the package.json complete:



{
  "name": "my-monorepo",
  "version": "0.0.0",
  "private": true,
  "workspaces": [
    "apps/*",
    "packages/*"
  ],
  "scripts": {
    "build": "turbo run build",
    "dev": "turbo run dev --parallel",
    "dev:app": "turbo run dev --filter=frontend",
    "lint": "turbo run lint",
    "format": "prettier --write \"**/*.{ts,tsx,md}\"",
    "build:app": "turbo run build --filter=frontend",
    "build:api": "turbo run build --filter=backend",
    "start:app": "turbo run start --filter=frontend",
    "start:api": "turbo run start --filter=backend",
    "heroku-postbuild": "if [ $CLIENT_ENV ]; then yarn run prod-frontend; elif [ $SERVER_ENV ]; then yarn run prod-backend; else echo no environment detected, please set CLIENT_ENV or SERVER_ENV; fi",
    "prod-frontend": "yarn run build:app",
    "prod-backend": "yarn run build:api"
  },
  "devDependencies": {
    "eslint-config-custom": "latest",
    "prettier": "latest",
    "turbo": "latest",
    "tsup": "^5.12.6"
  },
  "engines": {
    "npm": ">=7.0.0",
    "node": ">=8.0.0 <=16.14.2"
  },
  "dependencies": {},
  "packageManager": "yarn@1.22.18",
}
Enter fullscreen mode Exit fullscreen mode

🚨 I recommend not using the caches, but it's not a best practice; it's nice to study a better solution; I was facing issues keeping it true:



heroku config:set USE_YARN_CACHE=false -a frontend
heroku config:set NODE_MODULES_CACHE=false -a frontend
heroku config:set YARN_PRODUCTION=false -a frontend

heroku config:set USE_YARN_CACHE=false -a backend
heroku config:set NODE_MODULES_CACHE=false -a backend
heroku config:set YARN_PRODUCTION=false -a backend
Enter fullscreen mode Exit fullscreen mode

My turbo.json:



{
  "pipeline": {
    "build": {
      "dependsOn": ["^build"],
      "outputs": ["dist/**", ".next/**"]
    },
    "start": {
      "dependsOn": [
        "^build"
      ]
    },
    "start:app": {

    },
    "lint": {
      "outputs": []
    },
    "dev": {
      "cache": false
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Last but not least, you should run the deploy and see the result.

✅ Build and Deploy should pass. 🙏🏻

Conclusion

Excellent, you have a monorepo with turborepo running in production inside the Heroku.

Everything should now be proper to deploy multiple applications versioned under a monorepo to several Heroku applications.

Just set your Heroku applications up to deploy on push, and you should be ready to go next time you push any changes.

Always there is something to improve; what do I need to do? Github Actions, wait for the following chapters.

Finish ✌🏻

References:

Deploying a Monorepo to Heroku - by Sam

Pruning dependencies - Heroku Support NodeJS

javascript-monorepos

monorepo.tools

turborepo

__

Thanks for reading 🚀

Top comments (5)

Collapse
 
juni0r profile image
Andreas Korth •

Hey Thiago,

thanks for this article, it was a big help! I was wondering: you said, you're using pnpm. I assume you're using a custom buildpack on Heroku then? I'm using this one and it doesn't install dev dependencies on deploy (a sensible choice). As a result, I had to move turbo in dependecies.

Do you use any specifc setup with turbo/pnpm on top of what you described here? Maybe you want to share which buildpack you're using or some other useful tips?

Thanks!

Collapse
 
tgmarinhodev profile image
Thiago Marinho •

Hey Andreas, sorry for delaying.

But when deploying an app to Heroku, it is important to consider dependencies and configuration. By default, Heroku installs only production dependencies, ignoring development dependencies under devDependencies. To make Heroku install devDependencies, set the npm production variable to false using heroku config:set NPM_CONFIG_PRODUCTION=false

try this: heroku config:set NPM_CONFIG_PRODUCTION=false

Collapse
 
schroederg profile image
Alex Goncalves • • Edited

Oi, Thiago; Thank you so much for writing this article; it's awesome. I've followed along with everything you've outlined here, but encountered some issues. Specifically, upon deployment to Heroku, the build fails with the issue unknown flag: --filter=backend. Did you encounter this sort of trouble and if so, how did you resolve it?

Many thanks & muito obrigado,
Alex

Collapse
 
tgmarinhodev profile image
Thiago Marinho • • Edited

Thanks Alex, I'm using this version: "turbo": "1.6.3" and you? Also using pnpm: pnpm@7.14.2

Do you have this project on Github?

Collapse
 
sebpowell profile image
Sebastien Powell •

Hi Thiago,

Thanks for this article, very helpful. I followed the steps you outlined here, but running into an issue and wondering if you came across the same thing.

My repo structure looks like this:

|-- apps
|---/server
|---- Procfile
|---/frontend
|-- package.json
|-- pnpm-lock.yaml
|-- pnpm-workspace.yaml
|-- tsconfig.json
|-- turbo.json
Enter fullscreen mode Exit fullscreen mode

I am only deploying my server to Heroku, as I am using Vercel for the frontend. Worth noting as well that I am using pnpm. The buildpacks I am using are as follows (loaded in this order):

1.) github.com/heroku/heroku-buildpack...
2.) github.com/unfold/heroku-buildpack...
3.) github.com/heroku/heroku-buildpack...

Image description

I've also enabled the following environment variables:

NPM_CONFIG_PRODUCTION = false
PROCFILE = apps/server/Procfile
Enter fullscreen mode Exit fullscreen mode

The Procfile looks like this:

web: 'cd apps/server && yarn start:prod'
Enter fullscreen mode Exit fullscreen mode

My package.json looks like this:

{
  "name": "monorepo",
  "version": "0.0.0",
  "private": true,
  "workspaces": [
    "packages/*",
    "apps/*"
  ],
  "engines": {
    "node": "18.14",
    "npm": "9.8.1",
    "yarn": "1.22.19"
  },
  "scripts": {
    "build": "dotenv -- turbo run build",
    "build:backend": "turbo run build --filter=server",
    "heroku-postbuild": "pnpm run build:backend",
  },
  "devDependencies": {
    "dotenv-cli": "^7.2.1",
    "turbo": "latest"
  }
}
Enter fullscreen mode Exit fullscreen mode

And turbo, looks like this:

{
  "$schema": "https://turbo.build/schema.json",
  "globalDependencies": ["**/.env.*local"],
  "pipeline": {
    "build": {
      "dependsOn": ["^build", "^db:generate"],
      "outputs": [".next/**", "!.next/cache/**", "dist/**"]
    },
    "start": {
      "dependsOn": ["^build"]
    },
    "db:generate": {
      "cache": false
    },
    "db:migrate": {
      "cache": false
    },
    "db:push": {
      "cache": false
    },
    "lint": {},
    "test": {},
    "test:watch": {
      "cache": false
    },
    "dev": {
      "dependsOn": ["^db:generate"],
      "cache": false
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

When I push to Heroku, everything seems to work as it should, all the way until the end – this is the full output. One thing that is strange is it looks like it's building twice (?). If you have any ideas, they would be greatly appreciated – I have tried a bunch of different things, and none the wiser. Thank you in advance!

-----> Building on the Heroku-22 stack
-----> Using buildpacks:
       1. heroku-community/multi-procfile
       2. https://github.com/unfold/heroku-buildpack-pnpm
       3. heroku/nodejs
-----> Multi-procfile app detected
       Copied apps/server/Procfile as Procfile successfully
-----> Node.js app detected

-----> Creating runtime environment

       NPM_CONFIG_PRODUCTION=false
       NPM_CONFIG_LOGLEVEL=error
       NODE_VERBOSE=false
       NODE_ENV=production
       NODE_MODULES_CACHE=true

-----> Installing binaries
       engines.node (package.json):  18.14
       engines.npm (package.json):   9.8.1
       engines.yarn (package.json):  1.22.19
       engines.pnpm (package.json):  unspecified (use default)

       Resolving node version 18.14...
       Downloading and installing node 18.14.2...
       Bootstrapping npm 9.8.1 (replacing 9.5.0)...
       npm 9.8.1 installed
       Downloading and installing pnpm...
       pnpm 8.6.10 installed
       Resolving yarn version 1.22.19...
       Downloading and installing yarn (1.22.19)
       Installed yarn 1.22.19

-----> Restoring cache
       Cached directories were not restored due to a change in version of node, npm, yarn or stack
       Module installation may take longer for this build

-----> Installing dependencies
       Installing node modules (pnpm-lock.yaml)
       Scope: all 8 workspace projects
       Packages are hard linked from the content-addressable store to the virtual store.
         Content-addressable store is at: /tmp/codon/tmp/cache/.pnpm-store/v3
         Virtual store is at:             node_modules/.pnpm
       .../node_modules/@prisma/engines postinstall$ node scripts/postinstall.js
       .../node_modules/contentful postinstall$ node print-beta-v10-message.js
       .../node_modules/@nestjs/core postinstall$ opencollective || exit 0
       .../node_modules/@carbon/icons-react postinstall$ carbon-telemetry collect --install
       .../node_modules/contentful postinstall:   ---------------------------------------------------------------------------------------------
       .../node_modules/contentful postinstall:   contentful.js - the contentful delivery API (library)
       .../node_modules/contentful postinstall:   🚨 We have just released contentful.js v10 in Beta with enhanced ✨ TypeScript ✨ support! 🚨
       .../node_modules/contentful postinstall:   You can check it out on npm under the beta-v10 tag (go to the "Versions" tab to find it). 
       .../node_modules/contentful postinstall:   The migration guide and updated v10 README and can be found on the beta-v10 branch.
       .../node_modules/contentful postinstall:   README: https://github.com/contentful/contentful.js/blob/beta-v10/README.md
       .../node_modules/contentful postinstall:   MIGRATION GUIDE: https://github.com/contentful/contentful.js/blob/beta-v10/MIGRATION.md
       .../node_modules/contentful postinstall:   BETA BRANCH: https://github.com/contentful/contentful.js/tree/beta-v10
       .../node_modules/contentful postinstall:   ---------------------------------------------------------------------------------------------
       .../node_modules/contentful postinstall: Done
       .../node_modules/bufferutil install$ node-gyp-build
       .../es5-ext@0.10.62/node_modules/es5-ext postinstall$  node -e "try{require('./_postinstall')}catch(e){}" || exit 0
       .../node_modules/@nestjs/core postinstall: Done
       .../node_modules/fast-folder-size postinstall$ node get-sysinternals-du.js
       .../node_modules/bufferutil install: Done
       .../es5-ext@0.10.62/node_modules/es5-ext postinstall: Done
       .../node_modules/styled-components postinstall$ node ./postinstall.js
       .../node_modules/utf-8-validate install$ node-gyp-build
       .../node_modules/styled-components postinstall: Done
       .../node_modules/fast-folder-size postinstall: Done
       .../node_modules/utf-8-validate install: Done
       .../node_modules/@prisma/engines postinstall: Done
       .../node_modules/@carbon/icons-react postinstall: Done
       .../esbuild@0.16.4/node_modules/esbuild postinstall$ node install.js
       .../prisma@5.0.0/node_modules/prisma preinstall$ node scripts/preinstall-entry.js
       .../turbo@1.10.12/node_modules/turbo postinstall$ node install.js
       .../prisma@5.0.0/node_modules/prisma preinstall: Done
       .../esbuild@0.16.4/node_modules/esbuild postinstall: Done
       .../turbo@1.10.12/node_modules/turbo postinstall: Done
       .../node_modules/@prisma/client postinstall$ node scripts/postinstall.js
       .../node_modules/@prisma/client postinstall: prisma:warn We could not find your Prisma schema at 'prisma/schema.prisma'.
       .../node_modules/@prisma/client postinstall: If you have a Prisma schema file in a custom path, you will need to run
       .../node_modules/@prisma/client postinstall: `prisma generate --schema=./path/to/your/schema.prisma` to generate Prisma Client.
       .../node_modules/@prisma/client postinstall: If you do not have a Prisma schema file yet, you can ignore this message.
       .../node_modules/@prisma/client postinstall: Done

       devDependencies:
       + dotenv-cli 7.2.1
       + turbo 1.10.12

       Done in 53.5s

-----> Build
       Detected both "build" and "heroku-postbuild" scripts
       Running heroku-postbuild

       > monorepo@0.0.0 heroku-postbuild
       > pnpm run build:backend


       > monorepo@0.0.0 build:backend /tmp/build_58b4e67e
       > turbo run build --filter=server

       • Packages in scope: server
       • Running build in 1 packages
       • Remote caching disabled
       database:db:generate: cache bypass, force executing 41bfa3b1015a6009
       contracts:build: cache miss, executing e578a334607fbfa6
       database:db:generate: 
       database:db:generate: > database@0.0.0 db:generate /tmp/build_58b4e67e/packages/database
       database:db:generate: > prisma generate
       database:db:generate: 
       contracts:build: 
       contracts:build: > contracts@1.0.0 build /tmp/build_58b4e67e/packages/contracts
       contracts:build: > tsc
       contracts:build: 
       database:db:generate: Prisma schema loaded from prisma/schema.prisma
       database:db:generate: 
       database:db:generate: ✔ Generated Prisma Client (5.0.0 | library) to ./../../node_modules/.pnpm/@prisma+client@5.0.0_prisma@5.0.0/node_modules/@prisma/client in 925ms
       database:db:generate: 
       database:db:generate: ✔ Generated Zod Prisma Types to ./prisma/generated/zod in 1.44s
       database:db:generate: You can now start using Prisma Client in your code. Reference: https://pris.ly/d/client
       database:db:generate: 
       database:db:generate: import { PrismaClient } from '@prisma/client'
       database:db:generate: const prisma = new PrismaClient()
       database:db:generate: 
       server:build: cache miss, executing 831edf861abaf662
       server:build: 
       server:build: > server@0.0.1 build /tmp/build_58b4e67e/apps/server
       server:build: > nest build
       server:build: 

        Tasks:    3 successful, 3 total
       Cached:    0 cached, 3 total
         Time:    12.286s 

       - pnpm (nothing to cache)

-----> Pruning devDependencies
       Skipping because NPM_CONFIG_PRODUCTION is 'false'

-----> Build succeeded!
-----> Node.js app detected

-----> Creating runtime environment

       NPM_CONFIG_PRODUCTION=false
       NPM_CONFIG_LOGLEVEL=error
       NODE_VERBOSE=false
       NODE_ENV=production
       NODE_HOME=/tmp/build_58b4e67e/.heroku/node
       NODE_MODULES_CACHE=true

-----> Installing binaries
       engines.node (package.json):  18.14
       engines.npm (package.json):   9.8.1
       engines.yarn (package.json):  1.22.19

       Resolving node version 18.14...
       Downloading and installing node 18.14.2...
       Bootstrapping npm 9.8.1 (replacing 9.5.0)...
       npm 9.8.1 installed
       Resolving yarn version 1.22.19...
       Downloading and installing yarn (1.22.19)
       Installed yarn 1.22.19

-----> Restoring cache
       Cached directories were not restored due to a change in version of node, npm, yarn or stack
       Module installation may take longer for this build

-----> Installing dependencies
       Prebuild detected (node_modules already exists)
       Rebuilding any native modules
       npm ERR! code 127
       npm ERR! path /tmp/build_58b4e67e/node_modules/.pnpm/@hookform+resolvers@3.1.1_react-hook-form@7.45.1/node_modules/@hookform/resolvers
       npm ERR! command failed
       npm ERR! command sh -c run-s build:src build && check-export-map && husky install
       npm ERR! sh: 1: run-s: not found

       npm ERR! A complete log of this run can be found in: /tmp/npmcache.7Da8F/_logs/2023-07-30T12_25_37_177Z-debug-0.log
-----> Build failed
Enter fullscreen mode Exit fullscreen mode

Read next

freecoderx profile image

经过2个多月终于把我的github Sponsor弄好

FreeCoderX -

mahbubumithu profile image

A Complete Markdown Tutorial: From Basics to Advanced

MD MAHBUB UDDIN -

farhatsharifh profile image

From Changes to Safe Keeping: Git Stash

Farhat Sharif -

mayank_variya profile image

Streaming Real-Time Data in Flutter: A Step-by-Step Guide for Developers

Mayank Variya -