You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hey Ben, I used your script for connection to my private Ollama instance. Thanks for that.
Anyway i figured out that the connection was not working because curl closed the connection directly and did not wait for the response of the AI server. I tried with curl-timeout without success for whatever reason.
Took me a while but then I made the changes below to the code and added a loop with a timeout.
Not 100% perfect as the loop always adds another complete second, which is maybe not needed but
after adding the loop everything works as expected.
Up to you if you want to add it to the code or leave it here so it will maybe help someone else.
I tried to keep it configurable and added it only for ollama for now.
`private function execOllama() {
// a few more ollama settings that maybe are helpful
// This option determines whether curl verifies the authenticity of the peer's certificate. A value of 1 means curl verifies; 0 (zero) means it does not.
$sslVerifyPeer = 1;
// Set this option to zero to switch to the default built-in connection timeout - 300 seconds. But you can also set this higher
$curlInitialConnectionTimeout = 0;
// This parameter is used for 2 things. It sets the CURLOPT_TIMEOUT to this seconds variable. But as this did not help solving my problem it will also be the waiting time in the loop for the response from the AI-Server. Sometimes CURLOPT_TIMEOUT is not working as expected as it looks like. Time is in seconds. I set the default to 30 seconds.
$aiWaitTimeout = 30;
$post_data = [
"model" => $this->Model,
// "max_tokens" => $this->MaxTokens, // not used in ollama
"options" => [
"temperature" => $this->Temperature
],
'stream' => false,
"messages" => [
[ "role" => "system", "content" => '', ],
[ "role" => "user", "content" => $this->Prompt, ]
],
];
$post_fields = json_encode($post_data);
curl_setopt($this->curl, CURLOPT_URL, $this->reqUrl);
curl_setopt($this->curl, CURLOPT_POST, true);
curl_setopt($this->curl, CURLOPT_POSTFIELDS, $post_fields);
curl_setopt($this->curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($this->curl, CURLOPT_SSL_VERIFYPEER, $sslVerifyPeer);
curl_setopt($this->curl, CURLOPT_TIMEOUT, $aiWaitTimeout);
curl_setopt($this->curl, CURLOPT_CONNECTTIMEOUT, $curlInitialConnectionTimeout);
curl_setopt($this->curl, CURLOPT_HTTPHEADER, [
'Content-Type: application/json',
'Authorization: Bearer ' . $this->ApiKey,
]);
//intitialise the timer
$aiWaitTimeoutCounter = 0;
do {
$this->rsp->Raw = curl_exec($this->curl);
$responseCode = curl_getinfo($this->curl, CURLINFO_RESPONSE_CODE);
// keep in mind that sleep returns 0 on success,
// that would evaluate to false without strict comparison
if ($aiWaitTimeoutCounter == $aiWaitTimeout) {break;}
// add 1 second to the aiWaitTimeoutCounter
$aiWaitTimeoutCounter++;
// sleep for another second
} while ($responseCode !== 200 && sleep(1) !== false);
curl_close($this->curl);
$this->rsp->RawDecoded = json_decode($this->rsp->Raw, true);
if (isset($this->rsp->RawDecoded['message']['content'])) {
$this->rsp->Data = $this->rsp->RawDecoded['message']['content'];
} else {
$this->rsp->Err = true;
}
}`
Kind regards
Steffen
The text was updated successfully, but these errors were encountered:
Hey Ben, I used your script for connection to my private Ollama instance. Thanks for that.
Anyway i figured out that the connection was not working because curl closed the connection directly and did not wait for the response of the AI server. I tried with curl-timeout without success for whatever reason.
Took me a while but then I made the changes below to the code and added a loop with a timeout.
Not 100% perfect as the loop always adds another complete second, which is maybe not needed but
after adding the loop everything works as expected.
Up to you if you want to add it to the code or leave it here so it will maybe help someone else.
I tried to keep it configurable and added it only for ollama for now.
`private function execOllama() {
Kind regards
Steffen
The text was updated successfully, but these errors were encountered: