Support string_view initialization and lookup in TfToken

[nvmkuruc]

Description of Change(s)

TfToken is designed to optimize storage and comparison of commonly used strings like property names, values, and other path components. It currently provides two constructors, one for NULL-terminated C-strings and the other for std::string.

There are cases where a token may need to be constructed from a part of a std::string. This often necessitates an intermediate heap-allocated copy when no allocation or copy is needed at all if the token already exists in the registry. The path parser currently optimizes this case by introducing a stack allocated char[32] and copying and NULL-terminating strings short enough to fit in the buffer. However, this has not been generalized outside of the path parser, is limited to short strings, and still requires a copy.

This change introduces a std::string_view constructor and associated utilities. It benefits the current implementation by unifying the C-string and std::string code paths, avoiding templating and specialization.

It also replaces Find(std::string const&) with the more general Find(std::string_view).

A note is left in the TfToken::_Rep class about using a set that supports transparent hashing. This would allow lookups to transparently compare std::string_view against a TfToken::_Rep without _Rep storing an extra std::string_view field.

Fixes Issue(s)

• I have verified that all unit tests pass with the proposed changes

• I have submitted a signed Contributor License Agreement

[jesschimein]

Filed as internal issue #USD-9588

[jesschimein]

/AzurePipelines run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[jesschimein]

/AzurePipelines run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[Denise-Yang]

Closing this PR out because this change causes _Rep to increase in size, and when we upgrade to C++23, we will be able to address the issue that this PR brings up without needing to bloat _Rep.