Stream: cull-os

Topic: Content addressed dynamic libraries

RGBCube (Jun 21 2024 at 20:10):

Extending ELF to link dynamic libraries via a content addressed URI

Could you elaborate? Sounds interesting.

RGBCube (Jun 21 2024 at 20:12):

If I'm understanding correctly, if that was possible it wouldn't matter where the dynamic libraries are, as long as the dynamic linker can find them? So you wouldn't have to rebuild the whole world if you decided to use /nux/store instead of /nix/store as the path isn't referenced anywhere in the ELF

RGBCube (Jun 21 2024 at 20:14):

It shouldn't be that much work to make it run, all that needs to work is the systems dynamic linker resolving the URI to a path on the FS. I wonder how it would affect other tools, though. Do binaries even rely on the "paths"(in this case, the uri that's served as a path) they dynamically loaded existing?

Tim DeHerrera (Jun 21 2024 at 21:19):

It's one of my crazier ideas, but hypothetically the binaries would have no reliance on the filepath at all, and could simply load from the URI if it exists (could still fall back on a path, potentially).

I've thought about this a bit, may need some explicit support from the file-system as well. I've wanted to look further into it for a bit though, get a more solid plan

RGBCube (Jun 21 2024 at 21:47):

Why would you need filesystem support? The dynamic linker could check for a prefix or a format match on the path field. Like if a library "path" starts with uri: it could treat that as a uri and find the equivalent path

Tim DeHerrera (Jun 21 2024 at 22:03):

Maybe not for binaries, but probably for interpreted languages. I'm actually not 100% sure on that either though. There might be a clever way to avoid it

Tim DeHerrera (Jun 21 2024 at 22:06):

I know someone gave a talk on a similar idea at nixcon maybe 2 years ago or so. Going over how one could extend ELF to suit Nix

Tim DeHerrera (Jun 22 2024 at 01:12):

The point of the fs being aware is mostly so that files can be stored via their content URI and not have any true file path whatever, and every file exposed is just a virtual path to a hard uri, roughly speaking

RGBCube (Jun 22 2024 at 13:58):

Ah I understand

RGBCube (Jun 22 2024 at 13:58):

Would this be possible to implement as a kernel module? without modifying every filesystem in existence

RGBCube (Jun 22 2024 at 14:00):

Perhaps something like /proc/$PID but for /file/$HASH (/file is up for debate, doesn't sound that good)

RGBCube (Jun 22 2024 at 14:13):

But then that's the Nix store we're talking about. Another level of indirection doesn't sound appealing

RGBCube (Jun 22 2024 at 14:32):

Tim DeHerrera said:

I know someone gave a talk on a similar idea at nixcon maybe 2 years ago or so. Going over how one could extend ELF to suit Nix

Do you have a link to this?

Tim DeHerrera (Jun 22 2024 at 18:32):

I tried to find it but failed. I should be able to track it down later when I have more time though. I'll post when I do

Last updated: Oct 18 2024 at 08:48 UTC