> ## Documentation Index
> Fetch the complete documentation index at: https://site.aspect.build/llms.txt
> Use this file to discover all available pages before exploring further.

# Publishing Bazel rules that depend on tools: take 2

> Learn how Aspect uses Bazel toolchains to automate Rust and Go binary releases, ensuring efficient cross-compilation and smooth user experiences.

export const BlogPost = ({title, date, authors, tags, image, children}) => {
  const tagList = tags ? tags.split(", ").filter(Boolean) : [];
  const tagSlug = t => t.toLowerCase().replace(/&/g, "").replace(/\+/g, "").replace(/[^a-z0-9]+/g, "-").replace(/^-+|-+$/g, "");
  const formattedDate = date ? new Date(date + "T00:00:00").toLocaleDateString("en-US", {
    year: "numeric",
    month: "long",
    day: "numeric"
  }) : "";
  return <section className="w-full flex justify-center px-4 py-12 md:py-16">
      <div style={{
    maxWidth: "800px",
    width: "100%"
  }}>
        {image && (typeof image === "string" ? <img noZoom src={image} alt={title} className="w-full rounded-xl mb-8" style={{
    maxHeight: "400px",
    objectFit: "cover"
  }} /> : <div className="blog-post-hero-image">{image}</div>)}
        <h1 className="text-3xl md:text-4xl font-bold text-zinc-900 dark:text-white">
          {title}
        </h1>
        <div className="flex flex-wrap items-center gap-3 mt-4 text-sm text-zinc-500 dark:text-zinc-400">
          {authors && <span>{authors}</span>}
          {authors && formattedDate && <span>·</span>}
          {formattedDate && <span>{formattedDate}</span>}
        </div>
        {tagList.length > 0 && <div className="flex flex-wrap gap-2 mt-3">
            {tagList.map(tag => <a key={tag} href={"/blog/tags/" + tagSlug(tag)} className="px-2 py-0.5 rounded-full text-xs bg-zinc-100 dark:bg-zinc-800 text-zinc-600 dark:text-zinc-400 hover:bg-blue-100 dark:hover:bg-blue-900/40 hover:text-blue-700 dark:hover:text-blue-300 transition">
                {tag}
              </a>)}
          </div>}
        <hr className="my-8 border-zinc-200 dark:border-zinc-700" />
        <div className="prose dark:prose-invert max-w-none">{children}</div>
      </div>
    </section>;
};

export const MarketingPage = () => <div className="marketing-page-marker" style={{
  display: "none"
}} />;

export const Section = ({children, className = "", gray = false, dark = false, id}) => <section id={id} className={`w-full flex justify-center px-4 py-16 md:py-24 ${gray ? "bg-gray-50 dark:bg-zinc-900" : dark ? "bg-zinc-900 dark:bg-zinc-950" : ""} ${className}`}>
    <div className="w-full" style={{
  maxWidth: "1140px"
}}>
      {children}
    </div>
  </section>;

<MarketingPage />

<BlogPost title="Publishing Bazel rules that depend on tools: take 2" date="2024-03-05" authors="Alex Eagle" tags="Releases, Bazel">
  <img noZoom src="https://mintcdn.com/aspectbuild/x1L7Iep716jCyJVo/images/blog/hashnode/hn-d2f49d1d0d.jpeg?fit=max&auto=format&n=x1L7Iep716jCyJVo&q=85&s=033a6c4a82fdd09f7cbbde5c2e4d2bfc" alt="" className="blog-post-cover" width="1260" height="833" data-path="images/blog/hashnode/hn-d2f49d1d0d.jpeg" />

  Previously I wrote about a pattern we developed for [https://github.com/aspect-build/bazel-lib](https://github.com/aspect-build/bazel-lib) to publish our Go binaries on each release, then use Bazel's [toolchains](https://bazel.build/extending/toolchains) support to fetch those on users machines. This ensures that the Go SDK and libraries are only a development dependency of bazel-lib, not leaked to users.

  We recently needed to follow this pattern in [https://github.com/aspect-build/rules\_py](https://github.com/aspect-build/rules_py) - but it didn't work quite the same way. That's because this time we wrote the tools in Rust, so that we could take advantage of some great OSS work from leaders in the Python ecosystem recently: [https://astral.sh](https://astral.sh) and [https://prefix.dev](https://prefix.dev). In particular, we wanted to use [https://docs.rs/rattler\_installs\_packages/latest/rattler\_installs\_packages/](https://docs.rs/rattler_installs_packages/latest/rattler_installs_packages/) to create our virtualenv.

  The difference between Go and Rust is in the support for cross-compilation. In Go, it's quite easy for every platform to compile the complete set of release binaries. As a result, we were able to check in the integrity hashes of the binaries at every commit of the repo. If a contributors changes a Go source file, they'll be forced to update the corresponding integrity hashes as part of their PR. As a result, every commit in the git history is "releasable" - the sources include the information needed to safely fetch pre-compiled binaries from the releases page.

  Rust is a bit trickier. This is both at the language level, and also because of how rules\_rust behaves. We were not able to get all the different cross-compiles working, so the source repo CANNOT always have the integrity hashes. Well... that's okay because we didn't really enjoy having to check those in anyway.

  ## The Fix

  After some debate, we formed the requirements:

  * The release instructions should still have one step: "push a tag to the repo". This ensures that developers don't push from local machine (non-reproducible, cannot make SLSA attestation that our binaries are built from the tagged sources, etc). It also reduces the maintenance burden which is critical for a small team maintaining lots of rulesets.

  * For packaging the ruleset, we're allergic to the complexity of teaching Bazel about a "tree of `filegroup` targets called `release_files` as that's too much maintenance work. We prefer just `git archive` especially because of `.gitattributes` support for things like "exclude the examples folder from the distribution" and "stamp the release tag into one of the source files".

  The solution: the GitHub Actions automation has to build the binaries, then we take those integrity hashes and modify the `.tar` file produced by `git archive` to insert them.

  Here's a quick code walk-through in case you need to build something similar:

  1. [https://github.com/aspect-build/rules\_py/blob/main/tools/release/BUILD.bazel#L31](https://github.com/aspect-build/rules_py/blob/main/tools/release/BUILD.bazel#L31) is an `sh_binary` that "delivers" the Rust tools to a `$DEST` folder

  2. [https://github.com/aspect-build/rules\_py/blob/main/.github/workflows/release.yml#L12-L38](https://github.com/aspect-build/rules_py/blob/main/.github/workflows/release.yml#L12-L38) ensures that when a release tag is pushed to the repo, GitHub Actions spins up a machine for each OS to run that "deliver" tool. Then, [https://github.com/aspect-build/rules\_py/blob/main/.github/workflows/release.yml#L46-L52](https://github.com/aspect-build/rules_py/blob/main/.github/workflows/release.yml#L46-L52) we fetch those artifacts to our release job.

  3. [https://github.com/aspect-build/rules\_py/blob/main/.github/workflows/release\_prep.sh#L16-L44](https://github.com/aspect-build/rules_py/blob/main/.github/workflows/release_prep.sh#L16-L44) The release script runs `git archive` like usual, but then we modify the `tools/integrity.bzl` file to contain the new hashes. Note that `git archive` HAS to be run in a pristine folder so that the release tag ends up stamped as our version in [https://github.com/aspect-build/rules\_py/blob/main/tools/version.bzl#L3-L5](https://github.com/aspect-build/rules_py/blob/main/tools/version.bzl#L3-L5). So we can't do any messing around with files until after we've run it.

  4. [https://github.com/aspect-build/rules\_py/releases/tag/v0.7.1](https://github.com/aspect-build/rules_py/releases/tag/v0.7.1) shows how an automated release looks after I pushed the `v0.7.1` tag. Inside the [`rules_py-v0.7.1.tar.gz`](https://github.com/aspect-build/rules_py/releases/download/v0.7.1/rules_py-v0.7.1.tar.gz) artifact, we have all our features:

     1. `tools/version.bzl` contains `_VERSION_PRIVATE = "v0.7.1"` thanks to [https://github.com/aspect-build/rules\_py/blob/78832372c4a8c17259882f9c27715f8ef6cf4451/.gitattributes#L8-L9](https://github.com/aspect-build/rules_py/blob/78832372c4a8c17259882f9c27715f8ef6cf4451/.gitattributes#L8-L9)

     2. the `examples` folder is absent, to keep the release artifact from getting huge as we pile on more example usage, thanks to [https://github.com/aspect-build/rules\_py/blob/78832372c4a8c17259882f9c27715f8ef6cf4451/.gitattributes#L5-L6](https://github.com/aspect-build/rules_py/blob/78832372c4a8c17259882f9c27715f8ef6cf4451/.gitattributes#L5-L6)

     3. `tools/integrity.bzl` contains the result of that release script, i.e. `RELEASED_BINARY_INTEGRITY = { "unpack-aarch64-apple-darwin": "12502bad22e0725baeb37b531322fe1d12dd053ae6716bcead88cad5e26f0dab",`\
        `...`

  5. Cool, now we just need to ensure that rules\_py users are setup to fetch those binaries rather than need rules\_rust. Thanks to the git-stamping in `version.bzl`, the macro users call can tell whether you're using a prerelease (developing on rules\_py or fetching a SHA of the repo: [https://github.com/aspect-build/rules\_py/blob/78832372c4a8c17259882f9c27715f8ef6cf4451/py/repositories.bzl#L61](https://github.com/aspect-build/rules_py/blob/78832372c4a8c17259882f9c27715f8ef6cf4451/py/repositories.bzl#L61)\
     This way users of a release get a different set of toolchains registered.

  6. That pre-built binary toolchain is just the usual incantation: one repo that contains `toolchain` calls for every platform: [https://github.com/aspect-build/rules\_py/blob/78832372c4a8c17259882f9c27715f8ef6cf4451/py/private/toolchain/repo.bzl](https://github.com/aspect-build/rules_py/blob/78832372c4a8c17259882f9c27715f8ef6cf4451/py/private/toolchain/repo.bzl) and then one repo for each platform that actually does the tool fetching: [https://github.com/aspect-build/rules\_py/blob/78832372c4a8c17259882f9c27715f8ef6cf4451/py/private/toolchain/tools.bzl](https://github.com/aspect-build/rules_py/blob/78832372c4a8c17259882f9c27715f8ef6cf4451/py/private/toolchain/tools.bzl). That ensures you only download tools for the platforms Bazel selected in toolchain resolution. Note that one tool is for the `exec` platform (we unpack wheels in an action) and the other is for the `target` platform (we create the virtualenv inside the `py_binary` runtime). Bazel handles this nicely, e.g. if you build a `py_image` on a Mac, you'll fetch the unpack binary only for darwin\_arm64 and the venv binary only for linux\_x86.

  7. Finally, we've got a test in [https://github.com/aspect-build/rules\_py/tree/main/e2e/use\_release](https://github.com/aspect-build/rules_py/tree/main/e2e/use_release) that asserts that the release works as we expected: the pre-built binary toolchains were selected, and we only downloaded tools for the toolchain-resolved platform.

  That's a pretty deep-dive into what we do in Bazel rules to provide an awesome end-user experience. More to come on rules\_py as we ship our 1.0!
</BlogPost>