RGBCube (Jun 21 2024 at 18:07): RGBCube (Jun 21 2024 at 18:07):Currently, I've only completed the tokenizer. But the general way I want to continue forward is like so:
That will be all that's done by the cab CLI tool. You will be able to import code from islands (e.g <github:cull-os/cab>) and create derivations (as well as convert islands into derivations). But you won't be able to import or inspect contents of derivations as Cab doesn't know how to build them, which is IMO a feature.
Srid (Jun 21 2024 at 18:10):Essentially, cull-os/cab provides its own language (not Nix) and cross-repo abstraction (not flakes) - but using the underlying .drv language, right?
RGBCube (Jun 21 2024 at 18:11):And then cab will invoke the configured derivation builder and copy the result to the current filesystem (if the builder is remote). The builder has to adhere to the "builder protocol"; something that I want to create and write a specification for. Unlike Nix or Guix, it will be documented and will allow for custom builders that accomplish custom needs. The specification will only require the builder to return a result matching the specified derivation, making it super flexible.
Srid (Jun 21 2024 at 18:11):FTR, I am able to use the tokenizer (had to use Nix shell from a different Rust project since shell.nix doesn't provide cargo and friends):
Srid (Jun 21 2024 at 18:12):RGBCube said:
- similar to Nix or Guix's .drv
Ah, wait, you said similar to. So nothing's shared with Nix/nixpkgs?
RGBCube (Jun 21 2024 at 18:13):Srid said:
Essentially,
cull-os/cabprovides its own language (not Nix) and cross-repo abstraction (not flakes) - but using the underlying.drvlanguage, right?
Yes, it's essentially just a simple tool to compose derivation files (though, I do not think I will use Nix's .drv syntax or even the .drv extension)
RGBCube (Jun 21 2024 at 18:14):Srid said:
RGBCube said:
- similar to Nix or Guix's .drv
Ah, wait, you said similar to. So nothing's shared with Nix/nixpkgs?
Yes, I'm not planning to be compatible. What Cab essentially is is a frontend to the Builder Protocol, it creates derivation files required by the protocol and communicates with the backend (the builder server) to produce results
RGBCube (Jun 21 2024 at 18:15):One could even use Nix for the (currently non existent) protocol to build things, as what Nix does it also create derivation files. Like another frontend
Srid (Jun 21 2024 at 18:15):I'm specifically wondering if it would be possible to use (ie., import <github:NixOS/nixpkgs>) the existing packages from nixpkgs in a Cull package file.
Andreas (Jun 21 2024 at 18:17):I am wondering if @Tim DeHerrera might not be interested in this. Because he and I talked about about separating evaluation and builder functions of Nix, potentially distributing them somehow.
RGBCube (Jun 21 2024 at 18:17):That would require Cab to be compatible with Nix syntax, so no. But if Nix ever gets support for the builder protocol, it wouldn't be that hard to use Nix alongside Cab to build things on your system (with the same daemon managing both)
Tim DeHerrera (Jun 21 2024 at 19:07):Definitely interesting, especially the builder protocol, as that as something I've been itching to improve for years now with Nix.
Have begun to wonder if a full scale rearch wasn't necessary. I've had some other more outlandish ideas like extending ELF to link dynamic libs via a content addressed URI instead of a file path
That way, you could make the file system layer a virtual abstraction, and maybe even present the files as a regular FHS while still retaining the benefits of a Nix build.
In any case, I'll keep a close eye on your project, and maybe even contrib at some point if it helps. Seems like you have a solid plan so far. Bravo :clap:
RGBCube (Jun 28 2024 at 12:47):Almost finished the AST, the macros are done for automatic code generation
RGBCube (Jun 28 2024 at 13:53):I'll continue monday, I'm done
RGBCube (Jul 03 2024 at 09:38):AST node and token structures are done, as well as the tokenizer and stuff. Only need to create a parser that creates them, which shouldn't be more than 600 loc
RGBCube (Jul 03 2024 at 09:39):Also I finally understand how rowan actually works, which will help
RGBCube (Jul 05 2024 at 15:33):
RGBCube (Jul 05 2024 at 15:33):Parsing and formatting for debug output works! Currently I haven't implemented much of the parser but the formatter is working nicely now
RGBCube (Jul 05 2024 at 15:34):I need to clean up a little bit, however. But that's for after the parser
RGBCube (Jul 09 2024 at 17:17):Found a much better way to write parsers, using chumsky. It's exactly like the megaparsec haskell library
RGBCube (Jul 09 2024 at 17:18):I'm going to ditch the current implementation and rewrite it all. Did I waste the last 10 days? absolutely not
RGBCube (Jul 09 2024 at 19:30):Actually no. I can't use chumsky because I need a lossless parser for the formatter. I'll continue doing this, it already parses pretty well
RGBCube (Jul 09 2024 at 19:33):Unless there's a way to do lossless parsing with chumsky... I'm so unsure right now
RGBCube (Jul 11 2024 at 10:45):
IT WORKS
RGBCube (Jul 11 2024 at 10:46):After this, everything should be easy
Tim DeHerrera (Jul 11 2024 at 17:00):lol, famous last words? :sweat_smile:
RGBCube (Jul 12 2024 at 11:58):
RGBCube (Jul 12 2024 at 12:00):
RGBCube (Jul 12 2024 at 12:00):image.png
meanwhile, nix without a recovering parser
RGBCube (Jul 12 2024 at 12:04):
Tim DeHerrera (Jul 12 2024 at 15:21):very nice!
RGBCube (Jul 23 2024 at 12:19):Parser is 100% complete!
RGBCube (Jul 23 2024 at 12:20):I implemented pratt parsing & appliation today and yesterday after a long while
RGBCube (Jul 23 2024 at 12:20):I should update the examples to reflect the design changes I did
RGBCube (Jul 25 2024 at 15:34):Cleaned up a lot of parsing code, wrote a fuzzer (which optionally saves valid code to test later for regressions) and checker
RGBCube (Jul 25 2024 at 15:34):https://x.com/RGBCubed/status/1816471661803127214
RGBCube (Jul 25 2024 at 15:34):Also wrote an sexpr formatter
Tim DeHerrera (Sep 18 2024 at 19:32):How goes it? Are you still actively developing this? Just curious because I have been building some things that may be useful for your (or any Nix like) system, but act as a more general format for them to leverage, rather than tightly coupling with a particular tool. I'd be interested to see if you'd find it useful here.
RGBCube (Oct 07 2024 at 07:57):I forgot to check this chat, but no I haven't had the chance to develop it further because of school. I have refined the design and direction I want to take the language and builder protocol though
RGBCube (Oct 07 2024 at 07:59):What have you created? I'm not really looking to use an older format because I want to use Cap'n Proto for this, and a single Cap'n file can be leveraged by lots of languages so it's a de facto standard
Andreas (Oct 10 2024 at 19:07):use Cap'n Proto for this
Believe it or not, we've I've had some discussions with Tim on that one as well :grinning_face_with_smiling_eyes: did you already build something with that?
RGBCube (Oct 11 2024 at 13:18):Nope, but it does seem pretty useful for creating APIs that can be used across multiple languages (thanks to the community) while enabling you to trivially extend the protocol
Last updated: Oct 18 2024 at 08:48 UTC