Skip to main content

​
In-memory caches

Bazel has a JVM server. When you shut it down, you lose caches. Action graph → analysis phase.

​
The Repository cache

This was introduced earlier in Caching fetches: the repository cache. This is entirely separate from the Action Cache, because only actions are hermetic and know all their inputs, and therefore a correct cache key can be computed. Note that it’s possible for the repository cache to be stored remotely and shared between computers, using --experimental_repository_cache_remote.

​
The Action Cache

The action cache is keyed on a hash of the details of spawning the action, such as the content hash of the tool binary, arguments passed, and environment variables. It’s the digestKey shown in the output of bazel dump --action_cache. Bazel guarantees you won’t get a “false positive” cache hit. The cache key is dependent on the execution platform. This makes it very hard to share intermediate results for mixed-architecture, such as between a MacOS development machine and a Linux CI system. Non-determinism will cause cache misses, because the cache key changes anytime a dependency produces a different output. Hash ordering and timestamps are typical sources of non-determinism.

​
Local

Bazel has an in-memory cache of actions previously executed, using the bazel-out folder as storage. Think of this as the “L1 cache”. However, Bazel doesn’t have a “memory” of previous outputs. When a file in bazel-out is replaced, then Bazel needs to run the action again to restore it. To have a more durable storage for the local cache, use the --disk_cache flag. Like an L2 cache.
Prior to Bazel 7.4, the Disk Cache grows unbounded because there was no eviction, see https://github.com/bazelbuild/bazel/issues/5139.

​
Remote

Most Bazel deployments will benefit immediately from adding a shared cache that multiple machines can access to lookup cache entries. If any other developer or CI instance has already executed a given action, the action can be looked up in this remote storage, and if present it can be downloaded to the execution machine if that file is needed as an input to execute an action. You might think of this as the “L3 cache”. Bazel will also download remote files to the bazel-out folder as required based on the target pattern you request to build, so you can access that output after the bazel command returns. See the --remote_download_minimal flag definition for details. The remote cache is divided into two storage areas:
  • The AC (action cache) holds only metadata, and is a map from the digestKey to the set of output hashes of files produced by the action
  • the CAS (content-addressable storage) is a map from Checksum of a file → the binary content of the file
Popular choices for the Remote cache are varied based on the complexity of your needs:
  • Simple: use a cloud storage bucket or a WebDAV server for each of the two storage areas. High latency.
  • Medium: a single-instance server that can optimize by serving “hot” content from memory. Aspect recommends https://github.com/buchgr/bazel-remote
  • Complex: a distributed, scalable and high-availability service, typically paired with a Remote Execution environment. Aspect recommends https://github.com/buildbarn
There are fancy setups where the action cache can be multi-region, and a miss in the local region causes a cross-region download. This is like an “L3 cache”. Google has an L2 cache on a mini-datacenter in each physical office location to speed up builds.