MLPBlock cannot support tuple input of GEGLU
#7994
Assignees
Comments
yiheng-wang-nv
added a commit
that referenced
this issue
Aug 7, 2024
Fixes #7994 . ### Description The current implementation does not support tuple input of "GEGLU" since it only change the out features of the first linear layer when the input is a string of "GEGLU". This PR enhances it, and also enable "vista3d" mode to support #7987 Tests are added to cover the changes. ### Types of changes <!--- Put an `x` in all the boxes that apply, and remove the not applicable items --> - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [ ] Breaking change (fix or new feature that would cause existing functionality to change). - [x] New tests added to cover the changes. - [ ] Integration tests passed locally by running `./runtests.sh -f -u --net --coverage`. - [ ] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [x] In-line docstrings updated. - [ ] Documentation updated, tested `make html` command in the `docs/` folder. --------- Signed-off-by: Yiheng Wang <vennw@nvidia.com> Signed-off-by: YunLiu <55491388+KumoLiu@users.noreply.github.com> Co-authored-by: YunLiu <55491388+KumoLiu@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
No description provided.
The text was updated successfully, but these errors were encountered: