Stream: services-flake

Topic: ✔ Require opinion on `example/llm`


view this post on Zulip Pol Dellaiera (Jun 14 2024 at 14:32):

So great, thanks !

view this post on Zulip Notification Bot (Jun 14 2024 at 14:44):

Shivaraj B H has marked this topic as resolved.

view this post on Zulip Pol Dellaiera (Jun 14 2024 at 14:47):

Oh crap. open-webui 0.3.4 just landed in nixpkgs-unstable.

view this post on Zulip Pol Dellaiera (Jun 14 2024 at 14:47):

@Shivaraj B H Please run again the flake.lock generation in example/llm so you'll get the newest open-webui release.

view this post on Zulip Shivaraj B H (Jun 14 2024 at 14:50):

@Pol Dellaiera we have automatic flake updates every week. It should get updated in 2 days (Sunday).

view this post on Zulip Pol Dellaiera (Jun 14 2024 at 14:50):

Even for subdirectories ?!

view this post on Zulip Shivaraj B H (Jun 14 2024 at 14:51):

Yes

view this post on Zulip Srid (Jun 14 2024 at 14:53):

Shivaraj B H said:

Pol Dellaiera we have automatic flake updates every week. It should get updated in 2 days (Sunday).

It suffers from this particular UX issue: https://github.com/DeterminateSystems/update-flake-lock/issues/82

view this post on Zulip Pol Dellaiera (Jun 14 2024 at 14:54):

Oh DetSys...

view this post on Zulip Srid (Jun 14 2024 at 16:36):

Srid said:

I propose this then,

Have two examples,

For video demo (on X and for docs), we can use the latter example.

On 2nd thought, I don't think we should do this. I'll open a PR

view this post on Zulip Pol Dellaiera (Jun 14 2024 at 16:39):

What do you have in mind?

view this post on Zulip Srid (Jun 14 2024 at 16:55):

https://github.com/juspay/services-flake/pull/228

view this post on Zulip Shivaraj B H (Jun 17 2024 at 15:34):

@Srid Why not have the ollama1 data dir at the top-level in ~/.services-flake instead of ~/.services-flake/llm? I ask this because we follow the top-level convention even in the data dir which is by default in $PWD

view this post on Zulip Srid (Jun 17 2024 at 15:36):

Shivaraj B H said:

Srid Why not have the ollama1 data dir at the top-level in ~/.services-flake instead of ~/.services-flake/llm? I ask this because we follow the top-level convention even in the data dir which is by default in $PWD

That makes sense. So ~/.services-flake/llm/data/...? Yes.

view this post on Zulip Shivaraj B H (Jun 17 2024 at 15:45):

No, I meant at the top-level itself. ~/.services-flake/<service>, in this case ~/.services-flake/ollama1

view this post on Zulip Shivaraj B H (Jun 17 2024 at 15:46):

(deleted)

view this post on Zulip Srid (Jun 17 2024 at 16:04):

Because we consider llm a self-contained (services-flake) "app", so it makes sense to have its own data directory.

view this post on Zulip Pol Dellaiera (Jun 20 2024 at 20:22):

image.png

view this post on Zulip Pol Dellaiera (Jun 20 2024 at 20:22):

(I'm back on Zulip again)

view this post on Zulip Pol Dellaiera (Jun 20 2024 at 20:23):

As I explained, I left here because I don't want to spread to too many communication platforms and because I'm really used to the UI in Zulip.

view this post on Zulip Pol Dellaiera (Jun 20 2024 at 20:24):

So I though I had deleted my account... but actually I succeeded to disable it, I don't know how.

view this post on Zulip Pol Dellaiera (Jun 20 2024 at 20:24):

Anyway, thanks for re-enabling it.

view this post on Zulip Pol Dellaiera (Jun 20 2024 at 20:34):

Pol Dellaiera said:

image.png

This is the PHI3 model here. It's small and I like it. It's the default LLM on my open-webui instance.

view this post on Zulip Pol Dellaiera (Jun 20 2024 at 20:37):

image.png
This is very impressive.

view this post on Zulip Pol Dellaiera (Jun 20 2024 at 20:47):

image.png

view this post on Zulip Pol Dellaiera (Jun 20 2024 at 20:48):

I don't know what you think guys.

view this post on Zulip Pol Dellaiera (Jun 20 2024 at 20:48):

But I'll check your messages tomorrow. Time to go to bed. Cheers.

view this post on Zulip Srid (Jun 21 2024 at 03:32):

One idea is to have the flake app take the model name(s) via env vars, overriding what’s in flake config.

view this post on Zulip Shivaraj B H (Jul 02 2024 at 14:00):

I just tried out phi3, it is quite nice and is only 2.4 G. Should we switch the example/llm to use this instead of the 9 G model?

view this post on Zulip Srid (Jul 02 2024 at 14:03):

You can make that call.


Last updated: Oct 12 2024 at 21:45 UTC