[BUG] Streaming chat for ChatOpenAI Custom

[yavisht]

Describe the bug
Streaming chat does not work on ChatOpenAI Custom

To Reproduce

Expected behavior
The messages are not streaming back from the chat widget, it just loads the whole message.

Screenshots

Screen.Recording.2023-10-18.at.12.27.44.pm.mov

Flow
See screenshot

Additional context
If I change it to ChatOpenAI, message are streaming back.

[billsecond]

Same here.

[lswencn]

I also encountered this issue. How can it be resolved?

[Wowso]

There is a workaround.

After the prediction API is executed, the Flowise server checks whether the Chat Model supports streaming through the isFlowValidForStream function.

The chatOpenAICustom model is missing from the streamAvailableLLMs variable.

By temporarily adding it, streaming can be enabled.

I'm not sure why only the chatOpenAICustom model is excluded.

Flowise/packages/server/src/utils/index.ts

Line 1188 in 56f9208

export const isFlowValidForStream = (reactFlowNodes: IReactFlowNode[], endingNodeData: INodeData) => {

const streamAvailableLLMs = {
        'Chat Models': [
            'azureChatOpenAI',
            'chatOpenAI',
            'chatOpenAI_LlamaIndex',
            'chatOpenAICustom', // Add
            'chatAnthropic',
            'chatAnthropic_LlamaIndex',
            'chatOllama',
            'chatOllama_LlamaIndex',
            'awsChatBedrock',
            'chatMistralAI',
            'chatMistral_LlamaIndex',
            'groqChat',
            'chatGroq_LlamaIndex',
            'chatCohere',
            'chatGoogleGenerativeAI',
            'chatTogetherAI',
            'chatTogetherAI_LlamaIndex',
            'chatFireworks',
            'chatBaiduWenxin'
        ],
        LLMs: ['azureOpenAI', 'openAI', 'ollama']
    }