Make "ai/" the default namespace

[ericcurtin]

So we can run commands like:

docker model run gpt-oss

without the "ai/".

This would be similar to ollama where the "library/" is typically not specified.

[ericcurtin]

Have to propagate this around a bit to "pull", etc. But going to wait what folks think about it

[ericcurtin]

Shortens this output by 3 characters also, can fit more in a terminal:

MODEL NAME         PARAMETERS  QUANTIZATION  ARCHITECTURE  MODEL ID      CREATED       SIZE
smollm2:135M-Q2_K  134.52 M    Q2_K          llama         eba11bf8f361  3 months ago  82.41 MiB
smollm2:135M-Q4_0  134.52 M    Q4_0          llama         f7e2fcce3b78  3 months ago  85.77 MiB

[ericcurtin]

I think I got everywhere

[ericcurtin]

@p1-0tr @xenoscopic @doringeman PTAL. Could someone click the build button for me please to check this passes?

[ilopezluna]

I think this is a great idea, thanks! I was just wondering if we should consider placing it deeper within model-distribution. I was thinking of a unified location where, given a string, we return a parsed name.Reference. That way, we could include logic like setting a default namespace in one place, without having to worry about missing it elsewhere.
WDYT?

[ericcurtin]

@ilopezluna you know the code better than me 😅 I've only read a few bits...

I do think in the implementation details, there should still be a "ai/" namespace for simplicity, in terms of storage etc. But it can be hidden from the user such as when someone runs:

docker model ls

I do think it's ok to return "ai/" in more advanced usages. Because that is the namespace, just hiding it from the common usages for UX reasons and nicer to use/read strings.

Kinda wish the 4 model-* repos were one repo. Would be easier to avoid breaking changes.

I haven't played around with model-distribution yet, I don't understand that component yet.

But at the deepest level, I think "ai/" should exist.

[kiview]

I think the idea is great from an UX perspective.
Regarding the implementation details, I defer to the team 🙂

Kinda wish the 4 model-* repos were one repo. Would be easier to avoid breaking changes.

Yeah, we are pondering the idea of merging the repos.

[ericcurtin]

"model-distribution" equivalent:

https://github.com/docker/model-distribution/pull/122/files

Still might want to keep the list change here, not sure

[xenoscopic]

I fully support the idea. @ekcasey Do you know the most Moby-analogous place to put this in our tooling? I'm not sure where library/ gets injected there.

Kinda wish the 4 model-* repos were one repo. Would be easier to avoid breaking changes.

1000% agree; I think we've decided this now internally, it's just a question of timeline.

[ericcurtin]

I fully support the idea. @ekcasey Do you know the most Moby-analogous place to put this in our tooling? I'm not sure where library/ gets injected there.

Kinda wish the 4 model-* repos were one repo. Would be easier to avoid breaking changes.

1000% agree; I think we've decided this now internally, it's just a question of timeline.

library/ is the Ollama equivalent of ai/. Since Ollama is 95% an OCI registry with a load of ggufs. It does raise the question, does docker model runner want to support pulling from Ollama? It would be easy to add:

docker model run ollama.com/smollm:135m

The reason why this may not be desired is Ollama used to use everyday ggufs, in recent times there are non-standard ggufs being pushed to the Ollama registry that don't work with upstream llama.cpp, namely these ones:

https://github.com/ollama/ollama/tree/main/model/models

For this reason I've started leaning pulling ggufs from dockerhub and huggingface. But ~90% of the Ollama ones will work.

Did a very similar change in lm-pull and RamaLama (which are basically the same code, lm-pull was a shorter example that was created first):

https://github.com/ericcurtin/lm-pull/blob/f688fb83fce2c96efeadb096b6cdaea11e133d4a/lm-pull.py#L263

[ilopezluna]

I think this is a great idea, thanks! I was just wondering if we should consider placing it deeper within model-distribution. I was thinking of a unified location where, given a string, we return a parsed name.Reference. That way, we could include logic like setting a default namespace in one place, without having to worry about missing it elsewhere. WDYT?

@ericcurtin I initially thought it would be better to implement it in a "deeper" place, specifically in model-distribution, so all clients of model-distribution would benefit of this.
However now I'm inclined to prefer to implement this Hub-specific behavior in the model-cli because I think it's the only client where it's needed, and this way model-distribution would not have to know about specifics of their clients.
I'm sorry because you have already implemented the model-distribution equivalent 🙇
So I vote to go with this approach, could you please resolve conflicts?

[kiview]

@ericcurtin @ilopezluna
Sorry for flip flopping this decision around like this 😅

I believe the right location is in the backend (i.e. model-distribution), because else every UI/Client has to implement it itself (or risk examples being not conceptually transferable):

• CLI
• Compose (once they would integrate through API instead of CLI, which IMO is the right integration)
• Every Testcontainers library
• The future Docker SDKs that are currently WIP

[xenoscopic]

I would second doing this in model-distirbution to stay aligned with the way that Moby does things (via the distribution library here, more specifically here).

[ericcurtin]

I'll see what I can do!

[ericcurtin]

@xenoscopic @ekcasey @ilopezluna could you pinpoint where in model-distribution this should be implemented? Looking at that codebase, it's not clear where I should start.

Also my simple check for '/' has it's problems in cases where one may use the sha256 checksum as a key with no slash. But I'm sure moby has this problem too, if somebody tries to name an image a sha256-compatible string.

[ericcurtin]

There still might be some client side cases... Like the API might return ai/ but we might want to mask that in ls output... Or maybe we do just mask it absolutely everywhere server-side and give clients no knowledge of "ai/"...

[ericcurtin]

There is some prior art here, normalizeHuggingFaceModelName is a very similar thing, take a string, change it, but it's done client-side. FWIW ollama implements this client side.

[ericcurtin]

@xenoscopic @ekcasey @ilopezluna @kiview

What about this one?

docker/model-distribution#126

Another thing I haven't figured out how does one test model-cli and model-distribution together.

[xenoscopic]

@ericcurtin I think that's a better place for it than the CLI, just left a few comments there w.r.t. where to add the change.