✔ Require opinion on `example/llm` · services-flake

I propose this then,

Have two examples,

examples/ollama-ui: basically our current example but with all of Pol Dellaiera 's suggestions (no custom dataDir; no pre-loaded models)

examples/llm: a new example that imports the previous example (shared service) and sets the two things only -- dataDir and models (here, we can use a good model without size restriction)

For video demo (on X and for docs), we can use the latter example.

On 2nd thought, I don't think we should do this. I'll open a PR

Pol Dellaiera (Jun 14 2024 at 16:39):

What do you have in mind?

Srid (Jun 14 2024 at 16:55):

https://github.com/juspay/services-flake/pull/228

Shivaraj B H (Jun 17 2024 at 15:34):

@Srid Why not have the ollama1 data dir at the top-level in ~/.services-flake instead of ~/.services-flake/llm? I ask this because we follow the top-level convention even in the data dir which is by default in $PWD

Srid (Jun 17 2024 at 15:36):

Shivaraj B H said:

Srid Why not have the ollama1 data dir at the top-level in ~/.services-flake instead of ~/.services-flake/llm? I ask this because we follow the top-level convention even in the data dir which is by default in $PWD

That makes sense. So ~/.services-flake/llm/data/...? Yes.

Shivaraj B H (Jun 17 2024 at 15:45):

No, I meant at the top-level itself. ~/.services-flake/<service>, in this case ~/.services-flake/ollama1

Shivaraj B H (Jun 17 2024 at 15:46):

(deleted)

Srid (Jun 17 2024 at 16:04):

Because we consider llm a self-contained (services-flake) "app", so it makes sense to have its own data directory.

Pol Dellaiera (Jun 20 2024 at 20:22):

image.png

Pol Dellaiera (Jun 20 2024 at 20:22):

(I'm back on Zulip again)

Pol Dellaiera (Jun 20 2024 at 20:23):

As I explained, I left here because I don't want to spread to too many communication platforms and because I'm really used to the UI in Zulip.

Pol Dellaiera (Jun 20 2024 at 20:24):

So I though I had deleted my account... but actually I succeeded to disable it, I don't know how.

Pol Dellaiera (Jun 20 2024 at 20:24):

Anyway, thanks for re-enabling it.

Pol Dellaiera (Jun 20 2024 at 20:34):

Pol Dellaiera said:

image.png

This is the PHI3 model here. It's small and I like it. It's the default LLM on my open-webui instance.

Pol Dellaiera (Jun 20 2024 at 20:37):

image.png
This is very impressive.

Pol Dellaiera (Jun 20 2024 at 20:47):

image.png

Pol Dellaiera (Jun 20 2024 at 20:48):

I don't know what you think guys.

Pol Dellaiera (Jun 20 2024 at 20:48):

But I'll check your messages tomorrow. Time to go to bed. Cheers.

Srid (Jun 21 2024 at 03:32):

One idea is to have the flake app take the model name(s) via env vars, overriding what’s in flake config.

Shivaraj B H (Jul 02 2024 at 14:00):

I just tried out phi3, it is quite nice and is only 2.4 G. Should we switch the example/llm to use this instead of the 9 G model?

Srid (Jul 02 2024 at 14:03):

You can make that call.

Last updated: Nov 15 2024 at 12:33 UTC