Models with a namespace (containing "/") couln't be used #42

New issue

Closed

opened 2024-11-06 14:55:10 +00:00 by skobkin · 3 comments

skobkin commented

2024-11-06 14:55:10 +00:00

Owner

Models with a namespace (containing "/") couln't be used.

They're fail model availability check with "Model not unavailable":

2024/11/06 17:59:46 INFO Selected models="{TextRequestModel:mannix/llama3.1-8b-lexi:q8_0 SummarizeModel:mannix/llama3.1-8b-lexi:q8_0}"
2024/11/06 17:59:46 INFO Checking models availability
2024/11/06 17:59:46 ERROR llm: Model request failed error="error, status code: 404, status: 404 Not Found, message: invalid character 'p' after top-level value, body: 404 page not found"
2024/11/06 17:59:46 ERROR Model not unavailable model=mannix/llama3.1-8b-lexi:q8_0

Most likely cause is / not being encoded as a part of the value and sent as it's a part of a URL.

Models with a namespace (containing "/") couln't be used. They're fail model availability check with "Model not unavailable": ``` 2024/11/06 17:59:46 INFO Selected models="{TextRequestModel:mannix/llama3.1-8b-lexi:q8_0 SummarizeModel:mannix/llama3.1-8b-lexi:q8_0}" 2024/11/06 17:59:46 INFO Checking models availability 2024/11/06 17:59:46 ERROR llm: Model request failed error="error, status code: 404, status: 404 Not Found, message: invalid character 'p' after top-level value, body: 404 page not found" 2024/11/06 17:59:46 ERROR Model not unavailable model=mannix/llama3.1-8b-lexi:q8_0 ``` Most likely cause is `/` not being encoded as a part of the value and sent as it's a part of a URL.

skobkin added the

bug

label 2024-11-06 14:55:10 +00:00

skobkin self-assigned this 2024-11-06 14:55:10 +00:00

skobkin commented

2024-11-06 15:09:47 +00:00

Author

Owner

Simple escaping using url.QueryEscape() isn't helping though.

Still getting 404 for mannix%2Fllama3.1-8b-lexi%3Aq8_0 (mannix/llama3.1-8b-lexi:q8_0).

Even with curl:

$ curl http://localhost:11434/v1/models/mannix%2Fllama3.1-8b-lexi%3Aq8_0                                                                                                                                                                
404 page not found%

Could be Ollama problem? But how then OpenWebUI works with that? 🤔

Simple escaping using `url.QueryEscape()` isn't helping though. Still getting 404 for `mannix%2Fllama3.1-8b-lexi%3Aq8_0` (`mannix/llama3.1-8b-lexi:q8_0`). Even with curl: ```shell $ curl http://localhost:11434/v1/models/mannix%2Fllama3.1-8b-lexi%3Aq8_0 404 page not found% ``` Could be Ollama problem? But how then OpenWebUI works with that? 🤔

skobkin commented

2024-11-06 16:06:13 +00:00

Author

Owner

After some investigation, I've created a bug report in Ollama: ollama/ollama#7528

After some investigation, I've created a bug report in Ollama: [ollama/ollama#7528](https://github.com/ollama/ollama/issues/7528)

skobkin referenced this issue from a commit

2024-11-06 16:40:05 +00:00

Using local search to check model existence (fixes #42).

skobkin commented

2024-11-06 16:40:09 +00:00

Author