http_kwargs not passed to HTTP.request

[kirill-kondrashov]

It seems that http_kwargs are not translated here properly:

create_chat("<token>", "gpt-3.5-turbo", [Dict("role" => "user", "content"=> "say something")];     http_kwargs = (verbose=2,))
OpenAI.OpenAIResponse{JSON3.Object{Vector{UInt8}, Vector{UInt64}}}(200, {
        "id": "chatcmpl-<...>",
    "object": "chat.completion",
   "created": <...>,
     "model": "gpt-3.5-turbo-0613",
   "choices": [
                {
                           "index": 0,
                         "message": {
                                          "role": "assistant",
                                       "content": "The world is full of endless possibilities and wonders waiting to be discovered."
                                    },
                   "finish_reason": "stop"
                }
              ],
     "usage": {
                     "prompt_tokens": 9,
                 "completion_tokens": 14,
                      "total_tokens": 23
              }
})

If the (verbose = 2,) is passed to HTTP.request directly, it will display the debug information in the REPL.

[kirill-kondrashov]

The test here only checks that the reply is successful, but in fact, if you put one more argument, for instance

        http_kwargs=(connect_timeout=10, readtimeout=0, verbose=2)

the test should throw:

julia> create_chat("<token>", "gpt-3.5-turbo", [Dict("role" => "user", "content"=> "say something")];     http_kwargs=(connect_timeout=10, readtimeout=0, verbose=2))
ERROR: MethodError: no method matching request(::String, ::String, ::Pair{Symbol, Int64}, ::Pair{Symbol, Int64}, ::Pair{Symbol, Int64}; body::IOBuffer, headers::Vector{Pair{String, String}})

Closest candidates are:
  request(::Union{Function, Type}, ::Any, ::Any, ::Any, ::Any) got unsupported keyword arguments "body", "headers"
   @ HTTP ~/.julia/packages/HTTP/e7yOj/src/HTTP.jl:455
  request(::Union{Function, Type}, ::Any, ::Any, ::Any, ::Any, ::Any; headers, body, query, kw...)
   @ HTTP ~/.julia/packages/HTTP/e7yOj/src/HTTP.jl:455
  request(::Any, ::Any, ::Any, ::Any; headers, body, query, observelayers, kw...)
   @ HTTP ~/.julia/packages/HTTP/e7yOj/src/HTTP.jl:313
  ...

Stacktrace:
 [1] request_body(url::String, method::String; input::IOBuffer, headers::Vector{Pair{String, String}}, kwargs::Base.Pairs{Symbol, Int64, Tuple{Symbol, Symbol, Symbol}, NamedTuple{(:connect_timeout, :readtimeout, :verbose), Tuple{Int64, Int64, Int64}}})
   @ OpenAI ~/.julia/dev/OpenAI/src/OpenAI.jl:80

[algunion]

It seems that http_kwargs are not translated here properly:

create_chat("<token>", "gpt-3.5-turbo", [Dict("role" => "user", "content"=> "say something")];     http_kwargs = (verbose=2,))
OpenAI.OpenAIResponse{JSON3.Object{Vector{UInt8}, Vector{UInt64}}}(200, {
        "id": "chatcmpl-<...>",
    "object": "chat.completion",
   "created": <...>,
     "model": "gpt-3.5-turbo-0613",
   "choices": [
                {
                           "index": 0,
                         "message": {
                                          "role": "assistant",
                                       "content": "The world is full of endless possibilities and wonders waiting to be discovered."
                                    },
                   "finish_reason": "stop"
                }
              ],
     "usage": {
                     "prompt_tokens": 9,
                 "completion_tokens": 14,
                      "total_tokens": 23
              }
})

If the (verbose = 2,) is passed to HTTP.request directly, it will display the debug information in the REPL.

I just did the following call:

create_chat(
        ENV["OPENAI_API_KEY"],
        "gpt-3.5-turbo",
        [Dict("role" => "user", "content" => "Summarize HTTP.jl package in a short sentence.")],
        http_kwargs=(connect_timeout=10, readtimeout=0, verbose=2)
    )

And this gets printed:

┌ Debug: Marking request body stream
└ @ HTTP.RetryRequest ~/.julia/packages/HTTP/nn2yB/src/clientlayers/RetryRequest.jl:44
┌ Debug: POST /v1/chat/completions HTTP/1.1
└ @ HTTP.StreamRequest ~/.julia/packages/HTTP/nn2yB/src/clientlayers/StreamRequest.jl:23
┌ Debug: client startwrite
└ @ HTTP.StreamRequest ~/.julia/packages/HTTP/nn2yB/src/clientlayers/StreamRequest.jl:24
┌ Debug: HTTP.Messages.Request:
│ """
│ POST /v1/chat/completions HTTP/1.1
│ Authorization: Bearer sk.........................................
│ Content-Type: application/json
│ Host: api.openai.com
│ Accept: */*
.....
.....

At least, I can conclude from here that the verbose keyword was properly passed. The DEBUG info is also printed if I only use http_kwargs=(verbose=2,).

[kirill-kondrashov]

I guess there's a typo in my original message, ';' instead of ',', thanks for noticing that, the issue closed.

[kirill-kondrashov]

@algunion Hmm, for me it still throws:

julia> using OpenAI: create_chat

julia> create_chat(
               ENV["OPENAI_API_TOKEN"],
               "gpt-3.5-turbo",
               [Dict("role" => "user", "content" => "Summarize HTTP.jl package in a short sentence.")],
               http_kwargs=(connect_timeout=10, readtimeout=0, verbose=2)
           )
ERROR: MethodError: no method matching request(::String, ::String, ::Pair{Symbol, Int64}, ::Pair{Symbol, Int64}, ::Pair{Symbol, Int64}; body::IOBuffer, headers::Vector{Pair{String, String}})

Closest candidates are:
  request(::Union{Function, Type}, ::Any, ::Any, ::Any, ::Any) got unsupported keyword arguments "body", "headers"
   @ HTTP ~/.julia/packages/HTTP/e7yOj/src/HTTP.jl:455
  request(::Union{Function, Type}, ::Any, ::Any, ::Any, ::Any, ::Any; headers, body, query, kw...)
   @ HTTP ~/.julia/packages/HTTP/e7yOj/src/HTTP.jl:455
  request(::Any, ::Any, ::Any, ::Any; headers, body, query, observelayers, kw...)
   @ HTTP ~/.julia/packages/HTTP/e7yOj/src/HTTP.jl:313
  ...

Stacktrace:
 [1] request_body(url::String, method::String; input::IOBuffer, headers::Vector{Pair{String, String}}, kwargs::Base.Pairs{Symbol, Int64, Tuple{Symbol, Symbol, Symbol}, NamedTuple{(:connect_timeout, :readtimeout, :verbose), Tuple{Int64, Int64, Int64}}})
   @ OpenAI ~/.julia/packages/OpenAI/jZ9Qc/src/OpenAI.jl:80

Julia 1.9.2, OpenAI 0.8.5

[kirill-kondrashov]

I've also tried with fresh install of Julia 1.7.2, seems to throw:

julia> VERSION
v"1.7.2"
(@v1.7) pkg> st
      Status `~/.julia/environments/v1.7/Project.toml`
  [e9f21f70] OpenAI v0.8.5
julia> using OpenAI: create_chat
julia> create_chat(
                      ENV["OPENAI_API_TOKEN"],
                      "gpt-3.5-turbo",
                      [Dict("role" => "user", "content" => "Summarize HTTP.jl package in a short sentence.")],
                      http_kwargs=(connect_timeout=10, readtimeout=0, verbose=2)
                  )
ERROR: MethodError: no method matching request(::String, ::String, ::Pair{Symbol, Int64}, ::Pair{Symbol, Int64}, ::Pair{Symbol, Int64}; body=IOBuffer(data=UInt8[...], readable=true, writable=true, seekable=true, append=false, size=113, maxsize=Inf, ptr=1, mark=-1), headers=["Authorization" => "Bearer sk-<...>", "Content-Type" => "application/json"])
Closest candidates are:
  request(::Union{Function, Type}, ::Any, ::Any, ::Any, ::Any) at ~/.julia/packages/HTTP/nn2yB/src/HTTP.jl:455 got unsupported keyword arguments "body", "headers"
  request(::Union{Function, Type}, ::Any, ::Any, ::Any, ::Any, ::Any; headers, body, query, kw...) at ~/.julia/packages/HTTP/nn2yB/src/HTTP.jl:455
  request(::Any, ::Any, ::Any, ::Any; headers, body, query, observelayers, kw...) at ~/.julia/packages/HTTP/nn2yB/src/HTTP.jl:313
  ...
Stacktrace:
 [1] request_body(url::String, method::String; input::IOBuffer, headers::Vector{Pair{String, String}}, kwargs::Base.Pairs{Symbol, Int64, Tuple{Symbol, Symbol, Symbol}, NamedTuple{(:connect_timeout, :readtimeout, :verbose), Tuple{Int64, Int64, Int64}}})
   @ OpenAI ~/.julia/packages/OpenAI/jZ9Qc/src/OpenAI.jl:80
 [2] _request(api::String, provider::OpenAI.OpenAIProvider, api_key::String; method::String, http_kwargs::NamedTuple{(:connect_timeout, :readtimeout, :verbose), Tuple{Int64, Int64, Int64}}, streamcallback::Nothing, kwargs::Base.Pairs{Symbol, Any, Tuple{Symbol, Symbol}, NamedTuple{(:model, :messages), Tuple{String, Vector{Dict{String, String}}}}})
   @ OpenAI ~/.julia/packages/OpenAI/jZ9Qc/src/OpenAI.jl:136
 [3] openai_request(api::String, api_key::String; method::String, http_kwargs::NamedTuple{(:connect_timeout, :readtimeout, :verbose), Tuple{Int64, Int64, Int64}}, streamcallback::Nothing, kwargs::Base.Pairs{Symbol, Any, Tuple{Symbol, Symbol}, NamedTuple{(:model, :messages), Tuple{String, Vector{Dict{String, String}}}}})
   @ OpenAI ~/.julia/packages/OpenAI/jZ9Qc/src/OpenAI.jl:163
 [4] create_chat(api_key::String, model_id::String, messages::Vector{Dict{String, String}}; http_kwargs::NamedTuple{(:connect_timeout, :readtimeout, :verbose), Tuple{Int64, Int64, Int64}}, streamcallback::Nothing, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
   @ OpenAI ~/.julia/packages/OpenAI/jZ9Qc/src/OpenAI.jl:297
 [5] top-level scope
   @ REPL[4]:1

[jamblejoe]

I get the same error as @kir19890817 on Windows 11

julia> versioninfo()
Julia Version 1.9.3
Commit bed2cd540a (2023-08-24 14:43 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: 48 × AMD Ryzen Threadripper 3960X 24-Core Processor
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-14.0.6 (ORCJIT, znver2)
  Threads: 1 on 48 virtual cores
Environment:
  JULIA_CUDA_NSYS = C:\Program Files\NVIDIA Corporation\Nsight Systems 2022.4.1\target-windows-x64\nsys.exe
  JULIA_EDITOR = code
  JULIA_NUM_THREADS =

and package version OpenAI v0.8.5. Further, when only using one or two keywords, no error is thrown, but the keywords do not seem to have any effect. E.g.

create_chat(
                             OPENAI_API_KEY,
                             "gpt-3.5-turbo", 
                             [Dict("role" => "user", "content" => "Summarize HTTP.jl package in a short sentence.")],
                             http_kwargs=(max_tokens=2,)
                         )

OpenAIResponse{JSON3.Object{Vector{UInt8}, Vector{UInt64}}}(200, {
        "id": "chatcmpl-87s6oouZcAu3tL0EZiyMVM1Aa2ZOz",
    "object": "chat.completion",
   "created": 1696886982,
     "model": "gpt-3.5-turbo-0613",
   "choices": [
                {
                         "index": 0,
                         "message": {
                                       "role": "assistant",      
                                       "content": "HTTP.jl is a Julia package that provides a convenient and efficient way to make HTTP requests and handle responses."
                                    },
                   "finish_reason": "stop"
                }
              ],
     "usage": {
                 "prompt_tokens": 19,
                 "completion_tokens": 22,
                 "total_tokens": 41
              }
})

shows signigicantly more than 2 tokens as answer.

[roryl23]

I'll take a look at this tonight or tomorrow

[svilupp]

I believe this could be closed, as it has been fixed by: #49