Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Fix data leakage in KV cache initialization (#1057)
### Description This PR fixes a data leakage issue that can occur between generators when buffer sharing is enabled. ### Motivation and Context Suppose you create generator A with one input id and run one iteration of the generation loop with it. Now you destroy generator A and create generator B with three input ids. The input KV caches for generator B before running one iteration contain the values in the output KV caches for generator A after generator A ran for one iteration. The data leakage can be stopped when initializing the KV caches by always setting the memory to zeros.
- Loading branch information