Utilize string_view initialization of TfToken in path parsing

[nvmkuruc]

Description of Change(s)

(Depends on #3053)

GetToken tries to minimize dynamic allocations during path parsing by utilizing a stack allocated char[32] that can be explicitly NULL-terminated. This optimization still involves a copy to the intermediate buffer and only works for short strings.

This changes GetToken to use the new std::string_view constructor for TfToken instead. When the components of the path already exist in the registry, no copies or allocations are required. We've seen performance gains of about 4-8% in a simple test case where all the path components already exist as tokens.

Fixes Issue(s)

• I have verified that all unit tests pass with the proposed changes

• I have submitted a signed Contributor License Agreement

[jesschimein]

Filed as internal issue #USD-9589

[jesschimein]

/AzurePipelines run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[Denise-Yang]

Hi @nvmkuruc! Thanks for proposing this change, however we are leaning towards closing this out for now.
While we do appreciate the spirit of this PR, our main concern is that it increases the size of _Rep due the proposed std::string_view member, and our suite of performance tests demonstrated more regressions than gains. Furthermore with C++23, we will not need the const *char and the proposed std::string_view members since we will no longer need to construct temporary strings when inserting. As a result, we are thinking of including this optimization once we move to C++23.

However if you are interested in adding a string_view API or if the performance results for parsing paths are very compelling for you, then we could consider adding a more limited API that just introduces a string_view constructor or consider replacing the const * char member with a type that does not cause _Rep to bloat.

[Denise-Yang]

Closing this PR out because this change causes _Rep to increase in size, and when we upgrade to C++23, we will be able to address the issue that this PR brings up without needing to bloat _Rep.