Add local models to non-streaming accept list

[MatthewKhouzam]

What it does

Allows local model orchestrators like GPT4All to be a back-end for Theia's AI features by allowing to configure disableStreaming for them

Starts to address issues/14413

How to test

• Configure a custom Open AI model
• Add disableStreaming: true
• Observe (or debug) that the non-streaming API is used

Example Open AI configuration (using the official Open AI endpoint):

        {
            "model": "gpt-4o",
            "id": "my-custom-gpt-4o",
            "url": "https://api.openai.com/v1",
            "apiKey": true,
            "disableStreaming": true
        }

Follow-ups

We need to set up a max_token as different orchestrators stop at different lengths.

Review checklist

• As an author, I have thoroughly tested my changes and carefully followed the review guidelines

Reminder for reviewers

• As a reviewer, I agree to behave in accordance with the review guidelines

[sdirix]

Thanks for this hotfix.

In general we should make this configurable per (custom) model with an additional attribute.

[JonasHelming]

It is not only "custom" models, o1-preview also does not support streaming at the moment.
In general it probably makes sense to allows custom parameters per model on a global level.

[sdirix]

Middleground suggestion to get this in quickly:

• For now we can manually keep the hard coded non-streaming list of Open AI models as we maintain them manually anyway. This keeps the nicer UI for Open AI models.
• For all custom models we add a disableStreaming attribute which is false by default

[sdirix]

Adapted the PR. @MatthewKhouzam can you check whether this works for you?

[sdirix]

@MatthewKhouzam Did you have a chance to look the changes? Can we merge?

[TheMatthew]

I am OK with this... but should it be enableStreaming... I'm imagining tech support saying "yeah, go ahead and enable disablestreaming." and people having a hard time.

[sdirix]

It's a bit more errorprone when a boolean is true by default as a simple Javascript check like if(foo.bar) will be falsy in case the value does not exist. However that should likely not dictate our user interface. So I'm fine with switching the naming and default around, i.e. enableStreaming which is by default true.

[MatthewKhouzam]

I want to be clear, I approved on my side, anything else left to do?