RGBCube (Oct 24 2024 at 16:54): RGBCube (Oct 24 2024 at 16:54):Fetching entire source trees using islands will be very inefficient. Say when we're doing:
packages.cargoBuild {
source = <github:cull-os/cab>,
}
The build derivation will depend on the source derivation. But we don't have a derivation, we only have an island.
Then what will we do? We will have to fetch it all and reupload it to our realizer, as a derivation (via toDerivation).
This could be fixed with fixed output derivations, but specifying hashes manually is cumbersome.
Instead of that, the implementation of toDerivation island will output a super tiny program (ideally written in Zig with -Osmall) that fetches the island and places the contents in a fixed output derivation.
There is one minor issue. There is no way to do cab build --update without instructing the realizer to re-run the fetcher program with the found hash instead of expected one. This will be terrible for huge downloads.
The fix is to have "self-fixing" derivations where the derivation will set its own fixed hash itself. Then we can have our "updating" program set its fixed hash and exit.
This will be done with a fifo in the programs working directory where it will write the final hash right before exiting.
RGBCube (Oct 25 2024 at 13:18):Fetching entire source trees using islands will be very inefficient. Say when we're doing:
packages.cargoBuild {
source = <github:cull-os/cab>,
}
The build derivation will depend on the source derivation. But we don't have a derivation, we only have an island.
Then what will we do? We will have to fetch it all and reupload it to our realizer, as a derivation (via toDerivation island).
This could be fixed with fixed output derivations, but specifying hashes manually is cumbersome.
Instead of that, the implementation of toDerivation island will output a derivation using a super tiny WASM file (which is shared for all builtin islands, and requires requiredRealizerFeatures = [ "wasm" ]) that fetches the island and places the contents in a fixed output derivation.
There is one minor issue. There is no way to do cab build --update (and fetch new islands without the flag) without instructing the realizer to re-run the fetcher program with the found hash after failing initially by running with a random one. This will be terrible for huge downloads.
The fix is to have "self-fixing" derivations where the derivation will set its own fixed hash itself. Then we can have our "updating" program set its fixed hash and exit.
This will be done with a fifo in the programs working directory where it will write the final hash in a number of formats (like our git fetcher can write the git revision at the start, before fetching and writing the Cab hash), and when each format is written the realizer has the right to terminate the derivation as it might already have it stored somewhere.
RGBCube (Oct 25 2024 at 13:20):(updated to say WASM, for truly system independent building)
RGBCube (Oct 25 2024 at 13:21):Which makes me think, does having system be something special make sense at all? Why not make it a feature in the protocol too?
Last updated: Nov 15 2024 at 12:33 UTC