Update backend/python/transformers/transformers_server.py

Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
mudler · Mar 26, 2024 · 18747ef · 18747ef
1 parent b2ffd14
commit 18747ef
Showing 1 changed file with 0 additions and 1 deletion.
diff --git a/backend/python/transformers/transformers_server.py b/backend/python/transformers/transformers_server.py
@@ -260,7 +260,6 @@ def Predict(self, request, context, streaming=False):
                                 do_sample=True,
                                 pad_token=self.tokenizer.eos_token_id)
             generated_text = self.tokenizer.batch_decode(outputs[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0]
-            yield backend_pb2.Reply(message=bytes(new_text, encoding='utf-8'))
         return backend_pb2.Reply(message=bytes(generated_text, encoding='utf-8'))
 
     def PredictStream(self, request, context):