On adoption of monorepo

Feb 19, 2020 • updated Dec 05, 2020 • ☕️ 4 min read

There is a trend that many companies and open-source projects are moving from multi-repo into monorepo. Monorepo is not new, Google and Facebook have been using it for years. So why mass adoption now? I believe it depends a lot on the maturity of build systems and tools.

Monorepo in a nutshell

Monorepo is a software development strategy where code for many projects is stored in the same repository, the repository is large in content size and number of files, commits, or branches.

You literally can throw any kinds of projects into one monorepo, loosely connected or tightly coupled, one or multiple languages, one or multiple dependency management tools.

Monorepo has many benefits like single source of truth (unified versions,simplified dependency managers), large-scale refactoring (atomic changes), better collaboration (less boundaries, wide code visibility, clear tree structure, flexible ownership), and fluid continuous integration and delivery.

However monorepo does increase the codebase complexity, build time, and effort invested in both code health and tooling.

Multi-language monorepos

To manage a multi-language monorepo, you have to use powerful and complicated monorepo-oriented build systems like Buck, Bazel, Pants, or Please.

These tools are great for enterprise-level projects but seem a bit overkill for individuals or small teams, following are main characteristics:

  • Support a wide variety of language platforms (Python, Java, Javascript, C++, Android, iOS, Go, etc.)
  • Support incremental builds with local and distributed caching, optimized dependency analysis, and parallel execution.
  • Highly scalable, can handle any codebase size, especially enterprise huge monorepos.
  • Use a build languages like Starlark or Python DSL.

Google is famous for its enormous monorepo, early Google employees decided to work with a shared codebase managed through a centralized source control system. This approach has served Google well for more than 16 years, and today the vast majority of Google’s software assets continues to be stored in a single, shared repository, managed by Bazel build system.

Facebook developed Buck and improved Mercurial to suit its main source repository is enormous—many times larger than even the Linux kernel with thousands of commits a week across hundreds of thousands of files.

Twitter uses a custom build of Git which includes patches enhancing it to better deal with very large repositories in the range of multiple gigabytes.

JavaScript monorepos

Notable package managers like Yarn, Npm, or Pnpm have built-in feature called workspaces aim to make working with monorepo easy. At minimum, you’ll be able to keep multiple related packages all together in a single repository, and test changes in an integrated environment, without continually re-linking.

Lerna operates in a higher level than above package managers’ workspaces but not close to a build system when it has no support for build languages, caching, action directed graphs, or incremental builds.

Rush helps to ensure that installs and builds are completely deterministic. If you define custom commands or options, they are strictly validated and documented as part of Rush’s command-line help.

Nx is a set of extensible dev tools for monorepo, has first-class support for many frontend and backend technologies like Angular, React, and Node.

Thanks to awesome work at Lerna, many big javascript-based open-source projects have adopted monorepo flawlessly:

  • babel - compiler for writing next generation JavaScript
  • create-react-app - create React apps with no build configuration
  • react-router - Declarative routing for React
  • jest - Delightful JavaScript Testing
  • storybook - Interactive UI component dev & test
  • primer - the design system that powers GitHub

Summary

Monorepo is a software development strategy where code for many projects is stored in the same repository. Keep in mind monolithic source control does not necessarily result in monolithic software. And microservices can play well with monorepo.

You have to use powerful and complicated monorepo-oriented build systems like Buck, Bazel, Pants, or Please to manage multi-language monorepos.

In case of pure JavaScript monorepos, use JavaScript monorepo tools like Lerna, Rush or Nx to manage pure JavaScript monorepos.

Working with monorepo gets complicated over time when its size grows. Use monorepos to increase velocity during early stages of product development.

Related Posts

How CSS works under the hood

Some of the most fundamental concepts which control how CSS is applied to HTML and how conflicts are resolved

On expectation of technical blogging

Starting a technical blog is easy but maintaining it for a long period of time is a pain in the neck

Node version managers

Node Version Manager is a tool that helps you manage and switch between different Node.js versions with ease

The benefits of blogging

While the blogging scene has developed over the last decade, the benefits of blogging are still plentiful