Crash on Stage.OpenMasked when masking to an instanced Prim

[alanblevins]

Description of Issue

Stage.OpenMasked will usually cause USD to segfault when the mask specifies a Prim that is an instance. This appears to be related to threading because:

• sometimes it does not happen
• the crash does not happen at all if USD is limited to a single thread with Work.SetConcurrencyLimitArgument(1)

The stack trace looks like this:

#0  0x00007ffff089b514 in pxrInternal_v0_8__pxrReserved__::SdfPathTable<pxrInternal_v0_8__pxrReserved__::PcpPrimIndex>::_Grow() ()
   from /usr/pic1/usdtest/lib/python/pxr/Usd/../../../libpcp.so
#1  0x00007ffff089b818 in pxrInternal_v0_8__pxrReserved__::SdfPathTable<pxrInternal_v0_8__pxrReserved__::PcpPrimIndex>::insert(std::pair<pxrInternal_v0_8__pxrReserved__::SdfPath, pxrInternal_v0_8__pxrReserved__::PcpPrimIndex> const&) ()
   from /usr/pic1/usdtest/lib/python/pxr/Usd/../../../libpcp.so
#2  0x00007ffff0892fde in pxrInternal_v0_8__pxrReserved__::PcpCache::_ComputePrimIndexWithCompatibleInputs(pxrInternal_v0_8__pxrReserved__::SdfPath const&, pxrInternal_v0_8__pxrReserved__::PcpPrimIndexInputs const&, std::vector<std::shared_ptr<pxrInternal_v0_8__pxrReserved__::PcpErrorBase>, std::allocator<std::shared_ptr<pxrInternal_v0_8__pxrReserved__::PcpErrorBase> > >*) ()
   from /usr/pic1/usdtest/lib/python/pxr/Usd/../../../libpcp.so
#3  0x00007ffff0893c50 in pxrInternal_v0_8__pxrReserved__::PcpCache::ComputePrimIndex(pxrInternal_v0_8__pxrReserved__::SdfPath const&, std::vector<std::shared_ptr<pxrInternal_v0_8__pxrReserved__::PcpErrorBase>, std::allocator<std::shared_ptr<pxrInternal_v0_8__pxrReserved__::PcpErrorBase> > >*) () from /usr/pic1/usdtest/lib/python/pxr/Usd/../../../libpcp.so
#4  0x00007ffff10cbdfe in pxrInternal_v0_8__pxrReserved__::UsdStage::_ComposeSubtreeImpl(pxrInternal_v0_8__pxrReserved__::Usd_PrimData*, pxrInternal_v0_8__pxrReserved__::Usd_PrimData const*, pxrInternal_v0_8__pxrReserved__::UsdStagePopulationMask const*, pxrInternal_v0_8__pxrReserved__::SdfPath const&) () from /usr/pic1/usdtest/lib/python/pxr/Usd/../../../libusd.so
#5  0x00007ffff10e6e2e in pxrInternal_v0_8__pxrReserved__::WorkDispatcher::_InvokerTask<std::_Bind<std::_Mem_fn<void (pxrInternal_v0_8__pxrReserved__::UsdStage::*)(pxrInternal_v0_8__pxrReserved__::Usd_PrimData*, pxrInternal_v0_8__pxrReserved__::Usd_PrimData const*, pxrInternal_v0_8__pxrReserved__::UsdStagePopulationMask const*, pxrInternal_v0_8__pxrReserved__::SdfPath const&)> (pxrInternal_v0_8__pxrReserved__::UsdStage*, pxrInternal_v0_8__pxrReserved__::Usd_PrimData*, pxrInternal_v0_8__pxrReserved__::Usd_PrimData const*, pxrInternal_v0_8__pxrReserved__::UsdStagePopulationMask const*, pxrInternal_v0_8__pxrReserved__::SdfPath)> >::execute() ()
   from /usr/pic1/usdtest/lib/python/pxr/Usd/../../../libusd.so
#6  0x00007fffee1bd9da in tbb::internal::custom_scheduler<tbb::internal::IntelSchedulerTraits>::local_wait_for_all (this=0x7fffede83e00, 
    parent=..., child=<optimized out>) at ../../src/tbb/custom_scheduler.h:469
#7  0x00007fffee1b7b82 in tbb::internal::arena::process (this=0x7fffdabfd080, s=...) at ../../src/tbb/arena.cpp:160
#8  0x00007fffee1b65f4 in tbb::internal::market::process (this=0x7fffedf27c80, j=...) at ../../src/tbb/market.cpp:674
#9  0x00007fffee1b2687 in tbb::internal::rml::private_worker::run (this=0x7fffedd5d100) at ../../src/tbb/private_server.cpp:275
#10 0x00007fffee1b28a9 in tbb::internal::rml::private_worker::thread_routine (arg=<optimized out>) at ../../src/tbb/private_server.cpp:228
#11 0x0000003c8b607aa1 in start_thread () from /lib64/libpthread.so.0
#12 0x0000003c8b2e8bcd in clone () from /lib64/libc.so.6

Steps to Reproduce

1. Clone this repo with example scene and scripts: https://github.com/alanblevins/USD_OpenMasked_crash_repro.git
2. python crash.py
3. Note that USD does not crash when masking to a non-instanced prim: python nocrash_not_instance.py
4. Note that USD does not crash when limited to one thread: python nocrash_one_thread.py

System Information (OS, Hardware)

RHEL 6.9

Package Versions

Verified with USD-0.8.4, USD-0.8.5

Build Flags

python build_scripts/build_usd.py /usr/pic1/usdtest --tests --alembic --katana --katana-api-location <redacted> --houdini --houdini-location <redacted> --maya --maya-location <redacted>

[jtran56]

Filed as internal issue #160884.