#777

WIP: Add nix flake for baochip targets

Draftsbellemwants to merge intofrom

Nix Flake for Reproducible Builds for Baochip targets

This PR adds Nix-based reproducible builds for Baochip firmware artifacts (dabao, baosec, bootloader, baremetal). Related to #57.

TODO:

  • Rebase against latest main branch

What is Nix?

Nix is a package manager that provides reproducible builds by isolating dependencies and pinning exact versions. This ensures builds are identical across different machines and over time.

What's Included

  • flake.nix with build derivations for all targets
  • Custom Rust toolchain (1.92.0 1.93.0) with Xous targets built via rust-xous-flake
  • nix develop shell with toolchains (riscv32imac-unknown-xous-elf & riscv32imac-unknown-none-elf)
  • nix develop .#nightly shell with Rust nightly for cargo fmt
  • GitHub Actions CI workflow (.github/workflows/nix.yml) — example run
  • Deterministic build verification (nix build --rebuild)

Installing Nix

curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install

This installs Nix with flakes enabled. For other options, see Determinate Systems Nix Installer.

Building Packages

# Build the default package (dabao-helloworld)
nix build

# Build specific packages
nix build .#dabao-helloworld
nix build .#baosec
nix build .#bootloader

Build outputs are placed in a result symlink.

Available Packages

CommandAliasDescription
nix build .#dabao-helloworld.#dabaoDabao hello world app
nix build .#baosec—Baosec application
nix build .#bao1x-baremetal-dabao.#baremetalBaremetal dabao
nix build .#bao1x-boot0—Bootloader stage 0
nix build .#bao1x-boot1—Bootloader stage 1
nix build .#bao1x-alt-boot1—Alternative boot 1
nix build .#bootloader—All bootloaders (boot0 + boot1 + alt-boot1)

Development Shell

# Enter dev shell with stable Rust 1.92.0 + xous targets
nix develop

# Use standard cargo commands
cargo xtask dabao helloworld

# For code formatting (nightly required)
nix develop .#nightly
cargo fmt --check
cargo fmt

Verifying Reproducibility

nix build --rebuild .#dabao-helloworld

This builds twice and compares results. If they differ, it reports an error.

Challenges Solved

Custom Rust target: The riscv32imac-unknown-xous-elf target is built from source using Nix via rust-xous-flake. Note: this is not a bootstrapped build—it uses the pre-built Rust 1.92.0 compiler.

No git in Nix sandbox: The following files are patched at build time (via substituteInPlace in flake.nix) to use environment variables instead of git commands:

  • tools/src/sign_image.rs — uses hardcoded version instead of SemVer::from_git()
  • xtask/src/versioning.rs — reads XOUS_VERSION env var instead of git describe
  • tools/src/swap_writer.rs — reads GIT_REV env var instead of git rev-parse

Reproducibility:

  • Set SOURCE_DATE_EPOCH=1, CARGO_INCREMENTAL=0, and --remap-path-prefix for deterministic builds
  • Modified xtask/src/versioning.rs to use SOURCE_DATE_EPOCH for the build timestamp instead of Local::now() (commit e30d405)

Dependency vendoring: Used Crane to vendor Cargo dependencies. The locales/ workspace required separate vendoring due to its own Cargo.lock.

Speeding Up Builds with Cachix (Optional)

First builds compile the Rust toolchain and can be slow. Cachix provides pre-built binaries.

Using the public cache (read-only, no account needed):

# Install cachix
nix profile install nixpkgs#cachix

# Enable the xous-core cache
cachix use xous-core

# Builds now download pre-built artifacts when available
nix build .#dabao-helloworld

To enable CI cache writes (maintainers):

  1. Create account at cachix.org
  2. (Optional) Create a Cachix organization for shared team ownership
  3. Create public cache (e.g., xous-core)
  4. Generate auth token with write access
  5. Add secret: gh secret set CACHIX_AUTH_TOKEN
  6. Uncomment authToken and warmup-cache sections in .github/workflows/nix.yml

A public cache is available at https://app.cachix.org/cache/xous-core (read-only for PRs from forks).

Tested

  • dabao-helloworld on hardware
  • boot1 on hardware
  • baosec build
  • bao1x-baremetal-dabao build
  • CI workflow

Caveats to Consider for Reviewers

  • Custom Rust target is not bootstrapped—uses pre-built Rust 1.92.0 1.93.0 compiler
  • Version string (xousVersion) requires manual updates to gitTag and gitTagRevCount in flake.nix when tags change
  • locales/Cargo.lock must be kept in sync separately from main Cargo.lock
  • Builds run with --no-verify flag (skips signature verification) like ci (.github/workflows/build.yml) does
  • Deterministic build check on CI (nix build --rebuild) runs but failures are non-blocking (continue-on-error: true)
  • Only dabao, alt-boot1, boot1 have been tested on hardware
  • xtask/src/versioning.rs modified to support SOURCE_DATE_EPOCH for reproducible timestamps
  • Consider setting up Cachix to speed up CI builds
  • Commit https://github.com/betrusted-io/xous-core/pull/777/commits/b1b05c1082bbe240fb504940075718f0d8247286 belongs to another PR is there just to help minimize cache usage on Github CI infra

Troubleshooting

"experimental features 'nix-command' and 'flakes' are disabled"

If you installed Nix without the Determinate Systems installer, add to ~/.config/nix/nix.conf:

experimental-features = nix-command flakes

Then restart your shell or run sudo systemctl restart nix-daemon (Linux).


Outdated section

Note: This section documents some parts of the development process and is kept for historical reference.

(click to expand)

Still needs some work, but should work to build dabao helloworld, dabao-console, and boot1.

To use it, run nix develop to enter the dev shell, and then cargo xtask commands to build uf2 artifacts.

Some challenges

In sandboxed environments for nix build, git is not available, and therefore image signing via xous-semver fails as it relies on git describe to get the version. The current workaround is to patch the code "dynamically" in flake.nix to read a hardcoded base version combined with the git revision which nix has access to (self.shortRev). This still does not exactly match the output of git describe and thus needs to be addressed properly.

  • Figure out how to get semver without git describe for nix build sandboxed environment where git is not available.

Current flake.nix does not yield reproducible builds. This can be tested with nix build .#dabao --rebuild for instance.

  • Figure out how to get reproducible builds. NOTE: some changes had to be made to xous-core code to replace a timestamp with UNIX epoch time. -- perhaps this could be avoided with an in-place substitution (?).

Also relate to the sandbox environment, which does not have git access, we need to vendor dependencies.

  • Vendor dependencies, with crane or some other tools.

Note that in order to make the above work (build without git access, in offline mode), Cargo lock files had to be updated, and we run the xtask commands with --offline and --no-verify.

CI Cache Over-usage Issue

  • Need to figure out how to fine-tune caching so that it does not max out the cache capacity (10 GB).

In the meantime work is being done on fork at https://github.com/sbellem/xous-core/tree/nix-troubleshoot.

Answer: Use cachix

A public cache has been setup at https://app.cachix.org/cache/xous-core. For PRs from forks it can only be used to pull (read-only). To support pushes (writes) betrusted-io would need to add an auth token.

Activity

17
sbellem9mo ago
fixed: Need to fix boot1 as `audit` does not show semver
$ tio /dev/ttyACM0 
[13:25:30.227] tio 3.9
[13:25:30.228] Press ctrl-t q to quit
[13:25:30.228] Connected to /dev/ttyACM0
audit
Board type reads as: Dabao
Boot partition is: Ok(PrimaryPartition)
Semver is: 
Description is: Towards Beta-0
Device serializer: 00000000-00000000-0000e500-e5000069
Public serial number: JMGSNA
UUID: a2e2cfac-6a5d1193-dd1e6ad5-594f9426
...

Update

Fixed by patching (just during nix build) xtask logic when git describe is called, instead we read an env var set in flake.nix.

$ tio /dev/ttyACM0 
[17:44:52.774] tio 3.9
[17:44:52.774] Press ctrl-t q to quit
[17:44:52.775] Connected to /dev/ttyACM0
audit
Board type reads as: Dabao
Boot partition is: Ok(PrimaryPartition)
Semver is: v0.9.16-9871-g631dc42
Description is: Towards Beta-0
...
sbellem9mo ago
fixed: Need to fix boot1 as `audit` does not show semver
$ tio /dev/ttyACM0 
[13:25:30.227] tio 3.9
[13:25:30.228] Press ctrl-t q to quit
[13:25:30.228] Connected to /dev/ttyACM0
audit
Board type reads as: Dabao
Boot partition is: Ok(PrimaryPartition)
Semver is: 
Description is: Towards Beta-0
Device serializer: 00000000-00000000-0000e500-e5000069
Public serial number: JMGSNA
UUID: a2e2cfac-6a5d1193-dd1e6ad5-594f9426
...

Update

Fixed by patching (just during nix build) xtask logic when git describe is called, instead we read an env var set in flake.nix.

$ tio /dev/ttyACM0 
[17:44:52.774] tio 3.9
[17:44:52.774] Press ctrl-t q to quit
[17:44:52.775] Connected to /dev/ttyACM0
audit
Board type reads as: Dabao
Boot partition is: Ok(PrimaryPartition)
Semver is: v0.9.16-9871-g631dc42
Description is: Towards Beta-0
...
sbellem9mo ago
fixed: Need to fix boot1 as `audit` does not show semver
$ tio /dev/ttyACM0 
[13:25:30.227] tio 3.9
[13:25:30.228] Press ctrl-t q to quit
[13:25:30.228] Connected to /dev/ttyACM0
audit
Board type reads as: Dabao
Boot partition is: Ok(PrimaryPartition)
Semver is: 
Description is: Towards Beta-0
Device serializer: 00000000-00000000-0000e500-e5000069
Public serial number: JMGSNA
UUID: a2e2cfac-6a5d1193-dd1e6ad5-594f9426
...

Update

Fixed by patching (just during nix build) xtask logic when git describe is called, instead we read an env var set in flake.nix.

$ tio /dev/ttyACM0 
[17:44:52.774] tio 3.9
[17:44:52.774] Press ctrl-t q to quit
[17:44:52.775] Connected to /dev/ttyACM0
audit
Board type reads as: Dabao
Boot partition is: Ok(PrimaryPartition)
Semver is: v0.9.16-9871-g631dc42
Description is: Towards Beta-0
...
sbellem9mo ago
fixed: Need to fix boot1 as `audit` does not show semver
$ tio /dev/ttyACM0 
[13:25:30.227] tio 3.9
[13:25:30.228] Press ctrl-t q to quit
[13:25:30.228] Connected to /dev/ttyACM0
audit
Board type reads as: Dabao
Boot partition is: Ok(PrimaryPartition)
Semver is: 
Description is: Towards Beta-0
Device serializer: 00000000-00000000-0000e500-e5000069
Public serial number: JMGSNA
UUID: a2e2cfac-6a5d1193-dd1e6ad5-594f9426
...

Update

Fixed by patching (just during nix build) xtask logic when git describe is called, instead we read an env var set in flake.nix.

$ tio /dev/ttyACM0 
[17:44:52.774] tio 3.9
[17:44:52.774] Press ctrl-t q to quit
[17:44:52.775] Connected to /dev/ttyACM0
audit
Board type reads as: Dabao
Boot partition is: Ok(PrimaryPartition)
Semver is: v0.9.16-9871-g631dc42
Description is: Towards Beta-0
...
Rahzaelreviewed9mo ago
Rahzael9mo ago

@bunnie I wasn't quite sure if you wanted me doing a review on this, but figured it couldn't hurt to give my two cents. I put some comments below on how the nix stuff works and where it's getting code from. I'm much less familiar with the github actions portion, though.

@sbellem I had some questions below, specifically in regards to the github actions, as I'm not very familiar with them and want to make sure I understand where outside code is being pulled in from.

sbellemreviewed9mo ago
sbellemreviewed9mo ago
sbellemreviewed9mo ago
sbellemreviewed9mo ago
sbellemreviewed9mo ago
Rahzaelreviewed9mo ago
Rahzaelreviewed9mo ago

This pull request is a draft

Merging is unavailable until it is marked ready for review.