[BUG]: Error with Ollama AI provider "unmarshal: invalid character 'p' after top-level value"

[ppatierno]

Checklist

• I've searched for similar issues and couldn't find anything matching
• I've included steps to reproduce the behavior

Affected Components

• K8sGPT (CLI)
• K8sGPT Operator

K8sGPT Version

v0.3.40

Kubernetes Version

v1.26.3

Host OS and its Version

Fedora 40

Steps to reproduce

I configured an Ollama backend by following documentation here https://docs.k8sgpt.ai/reference/providers/backend/ and running ...

k8sgpt auth add --backend ollama --model llama3.1 --baseurl http://localhost:11434/v1

(I tried with llama2 as well but the model should not matter here)

The I just followed the Getting Started guide by creating the broken pod but the analyze by using the ollama backend returns the following error:

k8sgpt analyze --explain --backend ollama
   0% |                                                                                                                                                                                                              | (0/1, 0 it/hr) [0s:0s]
Error: failed while calling AI provider ollama: unmarshal: invalid character 'p' after top-level value

Tried the same by using Azure OpenAI and everything works as expected:

k8sgpt analyze --explain --backend azureopenai
 100% |████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| (1/1, 35 it/min)        
AI Provider: azureopenai

0: Pod default/broken-pod()
- Error: Back-off pulling image "nginx:1.a.b.c"
Error: The error indicates Kubernetes is unable to pull the image "nginx:1.a.b.c" because the specified tag is invalid.

Solution:
1. Check the image tag for typos.
2. Use a valid version tag, e.g., "nginx:1.21.6".
3. Update your deployment configuration with the correct image tag.
4. Redeploy the application.

Expected behaviour

Using ollama as backend doesn't return any unmarshal error but just an explanation about the broken pod.

Actual behaviour

No response

Additional Information

No response

[matthisholleville]

Hi! Have you tried calling the backend directly at http://localhost:11434/v1 without using k8sgpt? The error might indicate that your backend isn't responding correctly.

[ppatierno]

@matthisholleville I have actually found the issue. The baseurl has to be http://localhost:11434 (without the /v1) and it's working fine with k8sgpt this way. It's also because if I try ollama to be called directly via cURL, you can do something like the following (from ollama doc) and as you can see, there is no /v1 in the url.

curl http://localhost:11434/api/generate -d '{
  "model": "llama3.1",
  "prompt":"Why is the sky blue?"
}'

FYI I am using ollama 0.3.0.
Maybe I can contribute to the just fixing the k8sgpt doc if you want?

[matthisholleville]

Great! We could indeed validate the URL during the configuration or before calling the AI.

[ppatierno]

@matthisholleville FYI I opened a PR on the docs here k8sgpt-ai/docs#119

[alansenairj]

k8sgpt auth remove --backends ollama
k8sgpt auth add --backend ollama --model llama3.1:latest --baseurl http://localhost:11434