OpenAI.Request.LogItBias

OpenAI.Request.LogItBias

Property declaration

 Dim LogItBias As JSONItem

Remarks

Modify the likelihood of specified tokens appearing in the completion.

Accepts a JSONItem that maps tokens (specified by their token ID in the GPT tokenizer) to an associated bias value from -100 to +100. You can use this tokenizer tool (which works for both GPT-2 and GPT-3) to convert text to token IDs. Mathematically, the bias is added to the logits generated by the model prior to sampling. The exact effect will vary per model, but values between -1 and +1 should decrease or increase likelihood of selection; values like -100 or +100 should result in a ban or exclusive selection of the relevant token.

As an example, you can pass {"50256": -100} to prevent the <|endoftext|> token from being generated.