Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enhance reshape operator to determine shape of its output at runtime. #8781

Closed
lcy-seso opened this issue Mar 6, 2018 · 0 comments · Fixed by #9008
Closed

enhance reshape operator to determine shape of its output at runtime. #8781

lcy-seso opened this issue Mar 6, 2018 · 0 comments · Fixed by #9008
Assignees
Labels

Comments

@lcy-seso
Copy link
Contributor

lcy-seso commented Mar 6, 2018

The current implementation of reshape operator changes shape of an input Tensor into a specified shape and the shape information is set by operator attribute. This means the shape information is required to know before the operator runs. This is not the situation for some NLP tasks.

For example:

  • in Transformer, a 3-D tensor with a shape [batch size, max sequence length, hidden dimension] is needed to reshape into a 2-D tensor with a shape [batch size, max sequence length $\times$ hidden dimension] .
  • Both batch size and max sequence length cannot be known before the operator runs.
  • The current implementation of reshape operator set the shape of its output at compile time not run time.
@lcy-seso lcy-seso added the NMT label Mar 6, 2018
@lcy-seso lcy-seso self-assigned this Mar 6, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant